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ABSTRACT 


This  thesis  suggests  measures  of  performance  MSC  could  use  to  monitor 
ongoing  improvements  in  the  Special  Mission  Oceanographic  Ship  program.  A 
literature  review  was  conducted  to  establish  the  importance  of  measures  and 
propose  a  process  for  their  development.  Interviews  with  NAVOCEANO  and 
COMNAVMETOCCOM  personnel  determined  customer  needs  and  their 
expectations  concerning  service  quality.  Using  this  input,  a  list  of  performance 
measures  was  synthesized.  Interviews  of  MSCLANT  and  MSCPAC  personnel 
determined  the  measures  currently  tracked.  The  developed  measures  were 
compared  to  the  measures  currently  tracked  to  reveal  holes  or  overlap.  Finally, 
specific  customer  issues  with  service  were  addressed.  Research  found  that  MSC 
does  not  currently  measure  performance  in  these  areas.  Proposed  measures  were 
highlighted  that  would  track  performance  improvement  in  these  areas. 

This  thesis  research  found  that  there  are  some  elements  of  performance 
important  to  the  customer  which  are  not  being  tracked  by  MSC.  Secondly,  it  is 
important  to  involve  MSC  persoimel,  their  customers  and  suppliers,  etc.  in 
identifying  and  developing  measures  for  improvement  monitoring.  Finally, 
measures  identify  problems  and  quantify  the  improvement  made  in  performance. 
A  measurement  system  alone  won’t  correct  problems. 
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I. 


INTRODUCTION 


This  chapter  provides  a  brief  background  on  the  U.S. 
Transportation  Command  (USTRANSCOM)  and  one  of  its  component 
commands,  Military  Sealift  Command  (MSC) .  It  then  discusses 
MSC's  reinvention  process  and  introduces  MSC's  Special 
Mission  Program.  The  research  objectives,  methodology  and 
scope,  and  the  thesis  organization  will  follow. 

A.  BACKGROUND 

1.  United  States  Transportation  Command  (USTRANSCOM) 

USTRANSCOM  is  the  single  manager  responsible  for  the 
Defense  Transportation  System.  It  is  a  unified  command, 
established  under  the  Goldwater  Nichols  Act  in  1987.  It  has 
three  component  commands:  Military  Sealift  Command  (Navy), 
Air  Mobility  Command  (Air  Force)  and  Military  Traffic 
Management  Command  (Army) .  MSC  provides  USTRANSCOM  common 
user  Sealift  Transportation  for  the  entire  Department  of 
Defense  (DoD)  and  operates  the  prepositioning  ships.  Air 
Mobility  Command  (AMC)  provides  the  airlift  services  for 
USTRANSCOM.  Finally,  Military  Traffic  Management  Command 
(MTMC)  manages  all  DoD  personnel  and  cargo  movement  overland 
and  provides  the  interface  between  the  DoD  shippers  and  the 
commercial  carriers.  [Ref.  1:  p.  9-11] 
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2 .  Military  Sealift  Command 

Military  Sealift  Command  is  responsible  to  USTRANSCOM 
for  providing  all  sealift  transportation  for  DoD.  MSC 
oversees  a  fleet  of  more  than  130  ships  in  three  separate 
forces;  the  Naval  Fleet  Auxiliary  Forces  (NFAF) ,  the 
Strategic  Sealift  Force  and  the  Special  Mission  Program. 

[Ref.  2:  p.  3]  NFAF  ships  provide  direct  fleet  support  for 
the  Navy's  combatant  ships,  allowing  them  to  extend  their 
stay  at  sea.  The  Strategic  Sealift  Force  deploys  and 
sustains  U.S.  military  forces  worldwide.  The  Special 
Mission  Program  provides  and  operates  ships  for  specialized 
military  purposes.  [Ref.  3] 

MSC  is  headquartered  in  Washington,  D.C.  and  has  area 
commands  located  in  Bayonne,  NJ  (MSCLANT) ;  Oakland,  CA 
(MSCPAC) ;  London,  (MSCEUR) ;  and  Yokohama,  Japan  (MSCFE) . 
Future  consolidations  under  the  Base  Realignment  and  Closure 
(BRAC)  process  include  moving  MSCLANT  to  Norfolk,  VA,  MSCPAC 
to  Pearl  Harbor,  HI  and  MSCEUR  to  Naples,  Italy.  There  are 
many  other  subcommands  and  offices  located  worldwide. 
Approximately  8,000  people  make  up  MSC's  workforce.  This 
includes  military,  civil  service  and  contract  mariners. 

[Ref.  1;  p.  7-8] 

MSC  is  a  Defense  Business  Operations  Fund  (DBOF) 
activity.  As  such,  it  does  not  receive  any  direct 
appropriation  funding.  A  revolving  fund  is  established  from 
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which  payments  are  made  for  costs  incurred.  The  fund  is 
replenished  by  billing  MSC's  customers  for  services.  DBOF 
activities  do  not  make  a  profit  but  have  a  goal  to  break 
even.  Shortfalls  or  surpluses  are  carried  forward  from  year 
to  year. 

3.  MSC's  Reinvention 

MSC  is  currently  involved  in  a  reinvention  process  in 
response  to  military  downsizing  and  increasing  pressure  for 
government  agencies  to  improve  efficiencies  and  reduce 
costs.  In  February  1996,  MSC  implemented  a  program  manager 
organizational  structure  designed  around  major  service 
lines.  This  structure  places  more  emphasis  on  the  customer. 
The  program  management  concept  provides  a  mission  specific 
group  of  people  and  resources  who  can  provide  quick  response 
and  increased  flexibility  in  meeting  the  customer's 
requirements.  [Ref.  2:  p.  3-5] 

The  following  vision  statements  serve  as  guidelines 
during  MSC's  restructuring  effort: 

•  To  provide  uniformly  high  customer  service; 

•  To  have  clear  communication  channels; 

•  To  clarify  and  align  accountability,  responsibility 
and  authority; 

•  To  streamline  and  eliminate  duplication; 

•  To  be  proactive; 

•  To  provide  uniformly  high  organizational  flexibility 
and  responsiveness; 
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•  To  pursue  growth  opportunities ; 

•  To  take  care  of  our  people.  [Ref.  2:  p.  5] 

4.  Special  Missions  Program 

MSC's  Special  Missions  Program  is  one  of  six  program 
areas  under  the  new  structure.  [Ref.  2:  p.  5]  The  program's 
mission  is  " [t] o  manage,  operate,  repair  and  maintain  MSC's 
fleet  of  Special  Mission  Ships  which  perform  various  special 
missions  for  the  Department  of  Defense  customers."  [Ref.  2: 
p.  13]  It  consists  of  29  ships,  including  10  TAGOS,  eight 
oceanographic  ships,  five  range/cable  ships  and  six 
chartered  ships  (Chouest  owned) .  Two  program  officers,  one 
for  TAGOS  ships  and  the  other  for  range/cable/oceanographic 
ships,  report  to  the  program  manager  (PM- 2) .  Various 
detachments  throughout  the  United  States  will  support  the  ' 
program  officers  in  the  daily  operations. 

B .  RESEARCH  OBJECTIVE 

This  thesis  will  develop  a  set  of  measurement  criteria 
to  monitor  the  improvements  made  in  MSC's  Special  Mission 
Oceanographic  Ship  program  under  the  new  program  management 
structure.  These  can  be  used  to  measure  the  efficiency  and 
effectiveness  of  operations  and  the  impact  of  organizational 
changes.  They  can  also  provide  an  efficiency  incentive  for 
the  program  management  staff.  Using  data  collected  from 
MSC's  customers,  an  independent  set  of  measures  will  be 
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developed  and  compared  with  MSC' s  current  measures.  The 
result  will  be  a  set  of  measures  which  can  be  used,  not  only 
for  the  oceanographic  ships,  but  potentially  across  all 
business  areas  of  the  MSC  Special  Mission  Program. 

The  goal  of  this  thesis  research  is  to  answer  the 
following  research  questions: 

Primary: 

What  appropriate  measures  of  performance  can  MSC  use  to 
monitor  ongoing  improvements  in  the  Special  Mission 
Oceanographic  Ship  program? 

Subsidiary : 

What  aspects  of  service  are  important  to  MSC's 
oceanographic  ship  customers? 

What  aspects  of  service  does  MSC  perceive  to  be 
important  to  its  customers? 

C .  RESEARCH  SCOPE 

A  literature  review  of  performance  measures  provides 
the  foundation  for  developing  four  measurement  criteria  to 
monitor  performance  improvement  in  MSC's  Special  Mission 
Oceanographic  Ship  Program.  The  study  identifies 
performance  measures  currently  used  by  MSC  and  proposes 
additional  measures  based  on  customer  service  preferences. 
This  thesis  is  limited  to  the  oceanographic  ships. 
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D .  METHODOLOGY 

The  goal  of  this  thesis  is  to  develop  performance 
criteria  which  can  be  applied  to  MSC's  Special  Mission 
program.  A  literature  review  establishes  the  importance  of 
measures  and  proposes  a  process  for  their  development . 
Information  was  collected  through  interviews  with  NAVOCEANO 
and  COMNAVMETOCCOM  personnel  to  determine  customer  needs  and 
their  expectations  concerning  service  quality.  Interviews 
with  MSCLANT  and  MSCPAC  personnel  determined  the  measures 
currently  tracked.  The  developed  measures  were  compared  to 
the  list  of  measures  currently  tracked  to  reveal  any  holes 
or  overlap.  Finally,  specific  customer  issues  with  service 
were  addressed.  Research  found  that  MSC  does  not  currently 
measure  performance  in  these  areas .  Proposed  measures  were 
suggested  to  track  performance  improvement  in  these  areas. 
Finally,  the  potential  for  applying  these  four  broad 
measurement  criteria  to  other  areas  of  the  Special  Mission 
Program  was  discussed. 

E.  ORGANIZATION  OF  THESIS 

This  thesis  is  organized  as  follows; 

Chapter  I :  Introduction 

Chapter  II:  Literature  Review 

Chapter  III ;  NAVOCEANO  and  Current  Metrics 
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Chapter  IV:  Proposed  Measures 

Chapter  V:  Summary,  Conclusions  and  Recommendations 

Chapter  I  introduces  MSC,  their  mission,  the  current 
reinvention  efforts  underway  to  increase  customer 
satisfaction  and  the  need  for  measures.  Chapter  II  presents 
a  literature  review  which  establishes  the  foundation  for  the 
performance  measures,  what  they  are,  their  importance, 
problems  with  the  traditional  financial  measures  and  the 
need  for  organizations  to  establish  new  performance  measures 
to  stay  competitive  in  today's  business  environment.  An 
overview  of  the  measurement  criteria  presented  by  various 
authors  is  discussed.  From  their  proposed  measures,  a  list 
of  four  broad  measurement  criteria  is  synthesized.  These 
categories  can  be  applied  to  MSC's  Special  Mission 
Oceanographic  Ship  Program.  Chapter  III  introduces 
NAVOCEANO,  its  mission  and  some  background  on  the  ships  that 
are  managed  by  MSC.  This  chapter  also  describes  what 
services  MSC  provides,  what  measures  MSCLANT  and  MSCPAC  are 
currently  tracking  and  concerns  with  regard  to  MSC  services. 
Chapter  IV  proposes  a  set  of  measures  to  track  improvements 
in  MSC's  management  of  Special  Mission  Oceanographic  Ships. 
Finally,  Chapter  V  provides  a  summary,  conclusions  and 
recommendations . 
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II.  LITERATURE  REVIEW 


A.  MEASURES  OVERVIEW 


Mankind  measures  because  there  is  an  inherent  human 

desire  for  feedback  -  how  did  we  do  [Ref.  4:  p.  141]? 

According  to  Sink  [Ref.  5;  p.  65] : 

If  we  want  to  know  something  about  a  particular 
phenomenon,  we  measure  certain  attributes:  its 
size,  color,  shape,  temperature,  magnitude, 
weight,  state,  quality,  and  so  forth.  If  our 
intent  is  casual  or  not  particularly  critical,  we 
do  not  spend  much  time,  effort,  or  resources  on 
measurement.  However,  if  the  particular 
phenomenon  is  of  great  interest,  then  typically  we 
attempt  to  be  precise  and  accurate  in  our  efforts 
to  measure  or  specify  its  characteristics. 

Sink  and  Tuttle  describe  the  measurement  process  as 

"deciding  what  constitutes  performance  and  then  tracking 

indicators  against  our  concepts  of  performance...."  [Ref.  4 

p.  144] .  They  also  state  that  "...measurement  is  a 

substitute  for,  or  can  at  least  enhance,  direct  observation 

and  " . . .measurement  can  give  us  visibility  where  it  might 

otherwise  be  difficult"  [Ref.  4:  p.  158]. 

The  purpose  of  this  chapter  is  to  build  the  foundation 
for  the  thesis  by  providing  a  background  of  what  various 
experts  say  about  measures  and  how  they  are  used.  A 
description  of  the  various  roles  measurement  can  fulfill 
follows.  The  need  for  new  measures  will  be  discussed  along 
with  what  many  authors  consider  to  be  the  shortcomings  of 
traditional  measurements.  The  following  section  presents 
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guidelines  which  can  aid  organizations  in  developing 
measures.  Finally,  the  measurement  criteria  proposed  by 
various  authors  are  reviewed.  The  summary  for  this  chapter 
will  introduce  a  set  of  general  measurement  categories  which 
will  be  used  to  shape  the  specific  measurement  categories 
for  MSC  in  Chapter  IV. 

B.  ROLES  OF  MEASUREMENT 

Measurement  plays  a  variety  of  roles  in  the  management 
process.  Sink  and  Tuttle  [Ref.  4:  p.  144]  describe  in 
detail  the  various  roles  measurement  can  take  on,  such  as; 
ensuring  strategy  is  implemented,  a  management  support 
system,  a  control  device,  and  for  improvement.  A  brief 
overview  of  Sink  and  Tuttle's  important  points  concerning 
each  of  these  roles  follows. 

1.  Measurement  to  Ensure  Strategy  is  Implemented 

Organizations  must  ensure  that,  when  developing  a 
measurement  system,  it  reflects  and  is  influenced  by  their 
business  strategies.  Developing  measures  which  are 
specifically  linked  to  the  organization's  goals,  objectives 
and  strategies  forces  those  responsible  for  implementation 
to  think  through  costs  and  benefits,  cause-and-ef feet 
linkages,  and  implications  of  the  strategy.  [Ref.  4:  p.  150] 
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2. 


Measurement  As  a  Management  Support  System 


A  management  control  system  provides  the  manager  the 
critical  information  required  for  deciding  what  to  do,  how 
to  do  it,  directing  and  evaluating  performance  and  finally, 
deciding  what  should  be  changed.  Measurement  is  the 
foundation  for  developing  these  support  systems.  The 
maturity  and  complexity  of  the  measurement  systems  are 
linked  to  the  complexity  of  the  system  being  managed.  [Ref. 
4:  p.  152] 

3 .  Measurement  as  a  Control  Device 

This  is  considered  to  be  the  most  familiar  and  widely 
used  method  for  measurement.  It  consists  of  measurement, 
evaluation,  and  intervening  or  exerting  influence  on  the 
thing  being  measured  in  an  attempt  to  control .  Control  is 
not  an  output  but  an  outcome  resulting  from  the 
intervention.  [Ref.  4:  p.  155] 

4 .  Measurement  for  Improvement 

Sink  and  Tuttle  believe  the  most  important  reason  for 
measuring  performance  is  to  support  and  enhance  improvement . 
Measurement  can  reveal  system  capabilities  and  what  expected 
performance  levels  are  statistically  probable  from  the 
organization's  processes  and  systems.  Sink  and  Tuttle  also 
state  the  following: 

Measurement  can  tell  us  where  we  need  improvement, 

it  can  help  us  to  prioritize  where  to  devote  our 


11 


energies  and  resources,  it  can  motivate,  it  can 
tell  us  when  we've  gotten  better,  and  it  is  a 
natural  and  inherent  part  of  the  management 
process.  Good,  high-quality  measurement  systems 
don't  just  happen;  they  must  be  designed  and 
developed  and  maintained.  Well -designed  and 
developed  measurement  systems  linked  to  a  business 
strategy  that  is  understood  and  accepted  can  drive 
constant  performance  improvement.  [Ref.  4:  p.  149] 


As  previously  mentioned.  Sink  and  Tuttle,  along  with 
other  authors,  consider  measurement  for  improvement  to  be 
the  most  important  role.  This  thesis  will  be  based  on  that 
premise . 


C.  THE  NEED  FOR  NEW  MEASURES 

Measures  have  taken  on  a  renewed  importance  in  recent 
years.  Businesses  faced  with  growing  global  competition  are 
restructuring  to  reduce  costs,  improve  process  management, 
and  increase  quality  and  customer  satisfaction.  To  succeed 
in  a  global  marketplace,  they  must  perform  as  well  as  the 
world's  best.  The  performance  level  of  the  world's  best  is 
constantly  rising  due  to  improvements  in  technology, 
communications,  and  learning. 

Many  types  of  organizations  that  are  entrenched  in 
empire  building  and  maintaining  the  status  quo,  such  as 
government,  health  care,  utilities,  and  transportation,  are 
now  facing  competition  due  to  deregulation  and  budget  cuts. 

A  critical  success  factor  for  all  of  these  organizations  is 
the  ability  to  accomplish  rapid  improvement  based  on 


12 


continuous  learning,  or  for  some,  a  step-change  in 
performance.  [Ref.  6:  p.  19] 

The  recent  sense  of  urgency  among  government  agencies 
to  become  more  competitive  also  reflects  Public  Law  103-62, 
the  Government  Performance  and  Results  Act  (GPRA) ,  signed  by 
President  Clinton  in  1993 .  GPRA  "mandates  full 
participation  by  all  agencies  in  strategic  planning  (mission 
statements  and  long  term  goals)  and  performance  measurement 
by  fiscal  year  1999"  [Ref.  7:  p.  7]  . 

At  the  same  time,  many  companies  report  a  need  to 
develop  key  performance  measures  to  augment  the  traditional 
quarterly  and  annual  financial  measures.  These  incorporate 
both  financial  and  non-f inancial  measures.  The  traditional 
accounting  and  financial  measures  were  designed  long  ago, 
not  to  run  businesses,  but  to  meet  financial  and  regulatory 
reporting  requirements  [Ref.  6:  p.  7] .  According  to  Dr. 

Ivor  S.  Francis,  director  of  the  Deming  Centre  International 
in  Australia: 

Traditional  measures  of  accounting  are  inadequate 
to  understand  how  a  company  creates  value. 

Companies  may  spend  vast  sums  accounting  for 
present  fixed  assets  which  may  represent  but  a 
small  proportion  of  the  present  financial  value  of 
the  company;  at  the  same  time,  the  company's 
accountants  may  ignore  those  intangible  assets 
which  determine  the  production  of  future  wealth. 
Measuring  performance  must  entail  measurement  of 
the  potential  for  future  performance,  not  last 
year's  performance.  [Ref.  6:  p.  13] 


Shortcomings  in  traditional  accounting-based  measures  are 
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listed  by  Brancato  : 


[I]n  comparison  to  key  measures  of  performance, 

traditional  accounting  based  measures: 

•  are  too  historical; 

•  lack  predictive  power; 

•  reward  the  wrong  behavior; 

•  are  focused  on  inputs  and  not  on  outputs; 

•  do  not  capture  key  business  changes  until  it  is 
too  late; 

•  reflect  functions,  not  cross-functional 
processes,  within  a  company;  and 

•  give  inadequate  consideration  to  dif f icult-to- 
quantify  resources  such  as  intellectual  capital . 
[Ref.  6:  p.  17] 


D .  DEVELOPING  MEASURES 


Most  of  the  authors  agree  that  measures  should  be  born 
out  of  strategy.  Organizations  need  to  first  define  their 
vision  or  goals.  They  should  identify  their  unique  business 
niche,  the  basis  of  their  competitive  edge,  and  things  they 
must  do  to  succeed.  Then,  through  an  evaluation  process, 
personnel  translate  the  organization's  strategic  objectives 
into  a  logically  consistent  set  of  performance  measures. 

The  authors  generally  agree  that  it  is  best  to  start 
with  a  blank  sheet  of  paper  in  order  to  ignore  all  of  the 
performance  measures  currently  tracked,  thereby  ensuring  a 
higher  success  rate.  They  may  return  to  some  of  their  list 
of  existing  measures  at  a  later  time,  if  necessary,  to  fill 
in  any  gaps  discovered.  [Ref.  6:  p.  35] 

Various  authors  have  provided  guiding  principles  to  use 
when  developing  performance  improvement  measures .  What 
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follows  is  an  overview  of  ideas  presented  by  Sink  and  Tuttle 
[Ref.  4] ,  Brancato  [Ref.  6] ,  and  Globerson  [Ref.  8] . 

Sink  and  Tuttle  present  16  guiding  principles  which 
they  view  as  important  to  compete  with,  "The  New 
Competition. "  The  four  that  are  presented  here  are 
considered  the  most  important.  Foremost,  they  state  that 
"measurement  cannot  be  used  to  drive  performance  improvement 
—the  driver  must  be  the  business  strategy  and  the 
performance  improvement  plan."  [Ref.  4:  p.  211]  To  support 
this  point  Sink  and  Tuttle  stress  that  the  largest  hurdle 
managers  must  overcome  is  to  accept  the  fact  that  to  remain 
competitive  in  today's  world  requires  constant  performance 
improvement.  Only  after  that  will  measurement  become  a 
necessary  tool  to  guide  successful  performance  improvement 
decision  making. 

The  second  principle  states  that  "acceptance  of  the 
measurement  process  is  essential  to  its  success  as  a 
performance  improvement  tool."  [Ref.  4:  p.  212]  The  key 
point  that  they  and  other  authors  have  echoed  is  that 
involvement  by  all  employees  in  the  planning  and 
implementation  process  will  enhance  acceptance,  provide  a 
more  relevant  set  of  performance  measures,  and  maximize 
performance  results . 

Sink  and  Tuttle's  third  principle,  "measure  what's 
important— not  what's  easy  to  measure"  stresses  the  need  to 


15 


measure  the  "right"  things.  [Ref.  4:  p.  212]  People 
perceive  important  things  to  be  what  is  measured  and  will 
direct  resources  and  performance  down  that  same  path. 

Perhaps  this  is  best  summed  up  another  way  by  Sink  and 
Tuttle  when  they  state  "you  get  what  you  inspect,  not  what 
you  expect"  [Ref.  4:  p.  142] . 

Finally,  Sink  and  Tuttle  recommend  "adopt  an 
experimental  approach  to  measurement  systems  for 
improvement."  They  quote  from  Tom  Peters'  book.  In  Search 
of  Excellence ,  to  encourage  organizations  to'  avoid  delaying 
implementation  of  measures  until  they  are  perfect.  To 
ensure  success  it  is  best  to  develop  the  best  measures 
possible,  try  them  out,  then  change  them  if  necessary.  [Ref. 
4:  p.  212] 

Brancato  [Ref.  6:  p.  40]  provides  a  number  of 
principles  collected  by  numerous  business  executives  who 
have  successfully  developed  performance  improvement 
measures ; 

•  Choose  measures  which  best  describe  the  outcome  or 
accomplishment  you  expect. 

•  An  organization  can  focus  on  what's  important  by 
limiting  the  number  of  measures. 

•  Limit  the  measurement  precision  to  only  those  items 
which  will  provide  a  payback. 

•  A  few  measures  to  cover  an  area  of  interest  will 
provide  a  more  accurate  picture  than  one  global 
measure . 

•  To  maximize  change  and  improvement,  develop  measures 
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that  are  group  oriented  and  avoid  linking  them  to 
individual  employee  performance  measures. 

•  Continually  raise  the  performance  standards. 

•  Don't  be  afraid  to  change  the  measures  as  strategy 
changes . 


Globerson  [Ref.  8:  p.  39]  provides  ten  factors  which  he 
considers  important  for  relevant  criteria  development : 

•  Developing  performance  criteria  based  on  the 
organization's  objectives  forces  managers  to  define 
them  in  concrete  terms . 

•  Management  is  able  to  conduct  performance  comparisons 
for  similar  areas  as  a  result  of  established  relevant 
performance  criteria. 

•  Involving  customers,  management  and  employees  in  the 
criteria  selection  process  is  key  to  ensuring  the 
selected  criteria  meet  the  needs  of  the  entire 
organization. 

•  Criteria  must  be  measurable  and  concise  in  order  to 
be  valid  and  useful. 

•  Criteria  must  be  appropriate  for  the  organizational 
unit  being  evaluated  to  be  valuable. 

•  Both  ratio  criteria  and  absolute  criteria  are 
necessary  when  evaluating  organizations. 

•  Although  objective  criteria  are  preferred  and 
reliable,  subjective  criteria  are  necessary  for 
measuring  service . 

•  The  measurement  must  be  reliable  to  ensure  consistent 
and  accurate  results. 

•  Precision  in  selecting  and  using  calculation  methods 
is  critical. 

•  Measurement  criteria  should  be  selected  based  on 
relevancy,  not  ease  of  use. 
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E.  MEASUREMENT  CRITERIA  /  CATEGORIES 

This  section  explains  the  relationship  between 
measurement  criteria  and  individual  attributes .  It  also 
reviews  the  measurement  criteria  categories  proposed  by- 
various  authors  as  well  as  the  reasons  why  these  are 
believed  to  be  important. 

Sink  and  Tuttle  [Ref.  4;  p.  136]  and  Kaplan  and  Norton 
[Ref.  9:  p.  72]  both  liken  an  organization's  performance 
measurement  criteria  to  instrument  clusters  in  an  aircraft 
cockpit  (ie:  aircraft  position,  communication,  navigation 
and  engine  performance)  and  the  elements  of  those  criteria 
to  the  individual  instruments  and  gauges.  Just  as  an 
instrument  cluster  informs  the  pilot  of  the  overall  engine 
status,  measuring  a  combination  of  individual  attributes 
will  give  the  manager  the  big  picture  on  that  aspect  of 
performance.  As  the  whole  instrument  panel  tells  the 
current  status  of  the  plane,  the  combination  of  performance 
criteria  helps  the  management  team  to  manage  effectively. 
Relying  on  only  one  instrument  or  measure  can  spell 
disaster . 

Similar  to  designing  an  aircraft  instrument  panel,  it 
is  important  to  design  a  measurement  system  by  starting  at 
the  general  level  and  work  towards  the  detailed.  This 
ensures  the  organization  has  a  conceptual  level  of 
understanding  to  guide  them  through  the  process.  Starting 
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at  a  lower  level  of  detail  greatly  increases  the  possibility 
of  disaster.  Once  the  criteria  has  been  defined  it  is  best 
left  up  to  those  actually  involved  with  the  process  to 
develop  the  individual  attributes  to  measure.  As  earlier 
stated,  wide  scale  involvement  in  the  measurement  process 
ensures  success  through  understanding  and  acceptance.  [Ref. 
4:  p.  137] 

Through  their  work  and  study  of  literature  on 
organizational  performance  over  the  past  ten  years.  Sink  and 
Tuttle  [Ref.  4:  p.  170]  make  the  following  observations: 

1.  There  is  no  consensus  as  to  performance 
criteria  for  organizational  systems. 

2 .  There  are  no  consensus  operational  definitions 
for  the  commonly  cited  performance  criteria 
such  as  effectiveness,  efficiency,  quality,  or 
productivity. 

3 .  Much  of  the  research  that  looks  at  the  impact 

of  various  improvement  strategies ...  is  difficult, 
at  best,  to  translate,  evaluate,  and  interpret 
because  there  is  no  consistently  applied 
concept  of  what  constitutes  organizational  systems 
performance. 

4.  ...The  result  has  been  confusion  in  the 
literature  and  in  practice  with  respect  to 
performance  measurement  and  improvement.^ 

These  observations  should  be  considered  while  reviewing  the 

general  performance  measurement  criteria  proposed  by  Sink 

and  Tuttle,  Globerson,  Brancato,  and  Kaplan  and  Norton. 


1.  Sink  and  Tuttle's  Performance  Criteria  Categories 


^This  is  due  to  a  weakness  in  the  availability  of 
conceptual  models  and  operational  definitions  provided  by 
academicians  and  practitioners  in  the  field. 
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sink  and  Tuttle  [Ref.  4]  adopt  the  seven  performance 
criteria  first  presented  by  Sink  [Ref.  5] .  The  performance 
criteria  are  effectiveness,  efficiency,  quality, 
productivity,  quality  of  work  life,  innovation,  and 
prof itability/budgetability .  According  to  the  authors,  this 
generic  listing  of  performance  criteria  is  complete,  however 
criteria  do  overlap.  They  do  not  maintain  that  these  are 
the  only  correct  criteria.  Their  goal  is  to  instill  more 
consistency  on  defining  organizational  performance. 

Managers  are  encouraged  to  develop,  define  and  implement 
their  own  criteria  to  fit  their  specific  organizations. 

[Ref.  4:  p.  171] 

Figure  1  is  a  diagram  from  Sink  and  Tuttle  [Ref.  4:  p. 
137]  which  describes  the  interrelationship  of  the 
organizational  system  and  the  seven  performance  categories. 
Sink  and  Tuttle's  rationale  for  the  proposed  seven 
performance  criteria  is  briefly  explained  below. 

a.  Effectiveness 

Effectiveness  focuses  on  the  output  and  outcome 
side.  Some  of  the  operational  issues  associated  with 
effectiveness  include:  did  we  do  the  right  things  according 
to  the  specifications,  did  we  get  all  the  right  things  done, 
and,  did  we  get  the  right  things  done  on  time.  The  most 
frequently  used  attributes  to  further  define  effectiveness 
are  timeliness  and  quality.  The  authors  state  that  the 
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Quality 


Figure  1 .  Interrelationship  of  the  Organizational  System  and 
the  Seven  Performance  Categories  Proposed  by  Sink  and 
Tuttle.  From  Ref.  4. 


operational  measure  for  effectiveness  is  actual  output  (AO) 
divided  by  expected  output  (EO) .  Effectiveness  is  closely 
tied  in  with  the  planning  process.  The  planning  process 
determines  goals  for  what,  when  and  how  well  to  produce 
outputs.  The  effectiveness  measure  compares  the  plan  and 
the  actual  output.  [Ref.  4:  p.  171]  [Ref.  5;  p.  42] 

b.  Efficiency 

Efficiency  is  related  to  resource  consumption  on 
the  input  side.  According  to  the  authors,  efficiency  is 
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defined  as  the  resources  expected  to  be  consumed  (REC) 
divided  by  resources  actually  consumed  (RAC) .  The  authors 
note  a  relationship  between  efficiency  and  effectiveness 
where  the  REC  value  is  linked  to  the  EO  value,  and  the  RAC 
value  is  linked  to  the  AO  value.  The  REC  values  are 
measured  through  forecasts  and  budgets,  while  the  RAC  values 
are  measured  by  the  cost  accounting  systems.  The  authors 
emphasize  that  "an  organization  can  be  effective  and  not 
efficient,  efficient  and  not  effective,  neither  effective 
nor  efficient,  and  still  survive."  [Ref.  4:  p.  172] 

c.  Quality 

Sink  [Ref.  5]  describes  quality  as  "...the  degree 
to  which  the  system  conforms  to  requirements, 
specifications,  or  expectations."  He  distinguishes  quality 
from  effectiveness  by  stating  quality  reflects  the 
attributes  or  characteristics  for  which  a  product  is 
designed  and  produced  (such  as,  will  the  product  or  service 
do  what  it  was  designed  to  do?) .  Sink  and  Tuttle  [Ref.  4] 
state  that  quality  is  critical  in  all  stages  of  an 
organization's  management  process  and  resource  flow: 
upstream,  input,  transformation  process,  output,  and 
downstream.  They  operationally  define  quality  by  using  six 
checkpoints,  one  quality  checkpoint  at  each  stage  of  the 
process  and  one  checkpoint  for  the  overall  quality 
management  process.  [Ref.  4:  p.  173] 
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Quality  Checkpoint  1:  Sink  and  Tuttle  define  this  as 
"...the  selection  and  management  of  upstream  systems,"  such 
as  selection  of  vendors  and  suppliers,  communicating  with 
customers  concerning  their  needs  and  requirements,  and 
working  with  upstream  systems  (vendors  and  suppliers)  to 
ensure  they  provide  what  is  needed  to  produce  a  quality 
product  [Ref.  4:  p.  173] .  Upstream  systems  can  be  either 
internal  or  external  to  your  organization.  [Ref.  4:  p.  173] 

Quality  Checkpoint  2:  This  quality  assurance  check 
ensures  your  organization  receives  what  it  needs  and  expects 
from  the  upstream  system.  It  is  very  difficult  to  correct 
defects  from  the  upstream  system.  Therefore,  it  is 
extremely  important  to  ensure  that  the  process  is  under 
control.  Statistical  measures  are  primarily  used  at  this 
checkpoint.  [Ref.  4:  p.  173] 

Quality  Checkpoint  3 :  This  checkpoint  ensures  that 
quality  is  built  into  the  product  or  service  during  the 
transformation  process.  This  requires  measurement  and 
evaluation  processes  and  management  support  systems  which 
promote  and  reward  building  quality  into  the  product.  [Ref. 
4:  p.  174] 

Quality  Checkpoint  4:  This  checkpoint  ensures  that  the 
organization's  output  meets  the  customer's  specifications 
and  expectations.  Quality  checkpoint  4  does  not  replace 
quality  checkpoint  3.  Similarly,  a  quality  control  check  on 
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the  output  side  does  not  ensure  that  quality  is  built  into 
the  product  or  service.  [Ref.  4:  p.  175] 

Quality  Checkpoint  5:  This  checkpoint  focuses  on 
downstream  systems.  It  measures  "the  proactive,  detailed 
understanding  of  what  your  customers  want,  need,  expect,  and 
demand,  as  well  as  how  they  are  reacting  to  the  delivery  of 
the  goods  and  services  you  are  providing."  [Ref.  4:  p.  177] 

Quality  Checkpoint  6 :  This  checkpoint  coordinates  the 
overall  quality  management  process.  The  customers  play  an 
important  part  in  designing  and  developing  both  the  input 
and  output  side  of  an  organization.  This  is  where  quality 
is  differentiated  from  effectiveness  and  efficiency;  it  is 
both  an  input  and  output  side  issue.  Therefore,  quality 
must  be  measured  and  managed  at  all  five  checkpoints.  [Ref. 
4:  p.  172] 

d.  Productivity 

Sink  and  Tuttle  [Ref.  4:  p.  180]  operationally 
define  productivity  as  organizational  outputs  divided  by 
organizational  inputs  over  a  given  period  of  time.  Although 
this  definition  is  simplistic,  managers  have  a  difficult 
time  measuring  productivity  due  to  four  operational  issues: 

•  Organizations  find  it  difficult  to  quantify  the 
"tangibility"  of  outputs.  Doubting  the  completeness 
of  the  numerator  in  the  productivity  equation,  the 
users  start  to  question  the  validity  of  the  ratios. 

•  An  inconsistency  arises  in  what  is  defined  as  outputs 
snd  inputs  when  there  is  a  failure  to  understand  what 
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is  the  productivity  measure  unit  of  analysis.  Taking 
the  time  to  perform  a  thorough  input /output  analysis 
prior  to  developing  measures  can  eliminate  this 
problem. 

•  When  measuring  inputs  and  outputs,  it  is  important  to 
correlate  the  unit  of  time  for  which  we  are  measuring 
performance.  This  is  called  defining  the  measurement 
scope . 

•  People  tend  to  define  productivity  in  the  narrow 
sense  (output  over  input)  but,  when  they 
operationalize  it,  they  treat  it  as  if  it  were  the 
very  broad  measure  of  performance.  Productivity  is 
only  a  just  part  of  the  total  picture.  [Ref.  4:  p. 
180] 


e.  Quality  of  Work  Life  (QWL) 

Quality  of  work  life  (QWL)  is  operationally 
defined  by  Sink  and  Tuttle  as  how  people  feel  about  various 
aspects  of  their  work  life,  such  as  pay  and  working 
conditions  [Ref.  4:  p.  182] .  This  performance  criteria 
addresses  the  people  in  the  organization  involved  in  the 
transformation  process.  Many  managers  believe  that,  if 
people  feel  positively  about  factors  they  believe  are 
important,  it  will  positively  influence  their  performance. 
This  translates  to  a  positive  organizational  performance. 

However,  Sink  and  Tuttle  argue  that  research  findings 
over  the  years  show  that  there  is  little  relationship 
between  attitudes,  feelings,  and  performance.  They  do 
believe  that  there  may  be  a  relationship  between  positive 
feelings  and  employee  rewards  from  the  organization.  Sink 
and  Tuttle  sum  up  by  saying  that,  although  there  is  no 
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evidence  to  support  a  simple  link  between  happy  people  and 
productive  people,  quality  of  life  in  the  organization  is 
critical  to  the  organization's  overall  performance.  [Ref.  4: 

p.  182] 

f.  Innovation 

Sink  and  Tuttle  operationally  define  innovation  as 
"the  creative  process  of  changing  what  we're  doing,  how  we 
are  doing  things,  structure,  technology,  products,  services, 
methods,  procedures,  policies,  etc.  to  successfully  respond 
to  internal  and  external  pressures,  opportunities, 
challenges,  threats."  [Ref.  4:  p.  183]  The  three  key 
aspects  of  this  definition  are  creativity,  change,  and  a 
successful  response.  The  authors  summarize  the  definition 
of  innovation  as  "the  creative  process  of  successfully 
changing  whatever  it  takes  to  survive,  compete,  grow,  and 
obtain  whatever  your  desired  outcomes  are."  [Ref.  4:  p.  183] 

g.  Profitability/Budgetability 

Profitability  is  used  to  define  the  bottom  line 
for  the  private  sector;  budgetability  defines  the  bottom 
line  for  the  public  sector.  Both  represent  the  relationship 
between  an  outcome  in  the  downstream  system  and  the  input . 
For  a  profit  center,  the  definition  of  profitability  is  "a 
measure  or  set  of  measures  that  relates  revenues  to  costs." 
[Ref.  4:  p.  185]  For  a  cost  center,  budgetability  is  "a 
measure  or  set  of  measures  of  the  relationship  between 
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budgets  and  agreed  upon  goals,  deliverables,  and  timeliness 
with  actual  costs  and  actual  accomplishments  and 
timeliness."  [Ref.  4:  p.  185] 

h.  Interrelationships  Between  the  Seven 
Performance  Criteria 

In  describing  the  interrelationships  between  the 
seven  performance  criteria,  the  authors  state  that 
management  should  "focus  on  effectiveness  first:  What  are 
the  right  things  for  us  to  be  doing?"  [Ref.  4:  p.  186] 

Next,  it  is  important  to  define  efficiency  and  quality  by 
asking  the  question  "What  resources  will  we  need  to  consume 
to  accomplish  those  'right'  things  and  what  are  the  quality 
specifications?"  [Ref.  4:  p.  186]  Productivity  will  follow 
as  long  as  the  first  three  performance  criteria  are  managed 
properly. 

Quality  of  work  life  (QWL)  and  innovation  act  as 
moderators  between  productivity  and  profitability/ 
budgetability .  If  managed  properly  QWL  and  innovation  can 
enhance  the  organization's  performance.  However,  poor 
performance  in  these  areas  can  mean  disaster  for  an 
organization.  The  authors  consider  profitability/ 
budgetability  to  be  a  near-term  outcome;  survivability, 
growth,  improvement,  and  excellence  are  long-term  outcomes. 

Because  different  organizations  and  managers  are 
expected  to  weigh  the  seven  performance  criteria 
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differently,  it  is  important  to  balance  all  seven 
performance  criteria  to  ensure  long  term  success.  [Ref.  4: 

p.  186] 

As  stated  earlier,  the  author's  seven  performance 
criteria  define  the  clusters  of  instruments  on  the 
organization's  instrument  panel.  They  help  the  management 
team  manage  effectively.  It's  up  to  the  organization  to 
develop  specific  indicators  for  each  individual  performance 
criteria . 

2.  Globerson's  Performance  Criteria  Categories 

Globerson  [Ref.  8]  believes  it  is  important  to 
establish  both  vertical  and  horizontal  criteria  when 
developing  performance  criteria.  A  detailed  breakdown 
follows  for  both  vertical  and  horizontal  criteria. 

a.  Vertical  Criteria 

Vertical  criteria  are  used  for  measuring  a 
specific  unit's  performance,  such  as  the  finance,  personnel, 
or  production  department.  Globerson  proposes  developing 
four  vertical  criteria:  outputs,  inputs,  productivity  and 
quality.  [Ref.  8:  p.  31] 

Output  criteria  "are  those  criteria  that  relate  to 
final  products  or  services  and  are  expressed  as  quantities 
of  output  per  unit  of  time" .  Globerson  suggests  using  both 
absolute  criteria  (units  of  output  per  unit  of  time)  and 
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relative  criteria  (rate  of  increase  in  the  output  per  unit 
of  time) .  Both  are  necessary  to  tell  the  whole  story.  [Ref. 
8:  p.  31] 

Input  criteria  can  be  defined  as  the  input  of  resources 
per  unit  of  time.  Labor,  materials,  equipment  and  capital 
are  usually  considered  resources  and  inputs.  [Ref.  8:  p.  32] 

Productivity  criteria  reflect  the  units  of  output 
produced  per  specific  input  units,  such  as  number  of  units 
per  machine  hour  or  number  of  finished  units  per  employee 
hour.  Organizations  can  improve  productivity  by  either 
increasing  the  output  while  using  the  same  amount  of 
resources  or  by  maintaining  the  same  output  using  fewer 
resources.  [Ref.  8:  p.  32] 

Quality  criteria  focus  on  the  specific  characteristics 
of  the  service  or  product,  not  the  productivity  or  input 
measures.  Usually,  quality  and  productivity  criteria 
compliment  each  other.  [Ref.  8:  p.  33] 

b.  Horizontal  Criteria 

Horizontal  criteria  evaluate  processes  which  may 
cross  departmental  boundaries  or  may  begin  and  end  in  the 
same  department.  Two  types  of  horizontal  criteria,  process 
criteria  and  resource  criteria,  are  discussed  here.  [Ref.  8: 
p.  33] 

Process  criteria,  are  frequently  conducted  over  a 
varying  number  of  departments.  They  are  usually  measured  on 
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the  basis  of  three  different  criteria:  quality,  cost  and 
time.  Quality  can  be  expressed  in  terms  of  meeting  given 
specifications  or  accuracy.  Cost  refers  to  the  expenditures 
of  resources,  including  labor,  energy,  equipment,  building 
and  capital.  Time  is  measured  as  the  period  required  to 
complete  the  process.  [Ref.  8:  p.  33] 

The  other  horizontal  criteria,  resource  criteria, 
includes  efficiency  and  the  ratio  between  resources  and 
outputs.  [Ref.  8:  p.  35]  Efficiency  can  be  further  broken 
down  into  equipment,  labor,  materials  and  operating  capital. 

Equipment  efficiency  is  often  called  equipment 
utilization.  It  is  defined  as  the  percent  of  output 
produced  by  the  equipment  compared  to  the  maximum  output 
capacity  per  the  manufacturer's  specifications.  However, 
many  organizations  are  more  interested  in  the  inverse  of 
this  ratio,  which  is  called  "idle  time." 

Labor  efficiency  is  employee  output  divided  by  the 
standard  output  (output  produced  by  a  trained  employee 
working  at  a  normal  pace) .  Standard  output  is  determined 
using  work  measurement  methods.  Labor  efficiency  can  often 
exceed  100%  as  employees  gain  experience  and  learn  how  to 
improve  performance  over  time  (learning-by-doing) . 

For  material  efficiency,  inventory  turnover  is  a  common 
criteria.  Inventory  turnover  is  the  annual  value  of  all 
material  used  by  the  company  divided  by  the  current  value  of 


30 


the  material  presently  in  the  company.  Material  utilization 
compares  the  organization's  and  industry's  turnover  rate. 

The  more  an  organization  turns  over  its  inventory,  the 
higher  the  material  utilization  or  efficiency.  Operating 
capital  is  calculated  by  comparing  the  actual  interest 
received  with  that  of  the  maximum  interest  possible  for  that 
given  financial  investment.  [Ref.  8:  p.  36] 

The  ratio  of  resources  to  output,  is  the  second 
resource  criteria.  It  is  used  to  evaluate  resource 
performance  for  labor,  materials,  equipment  and  operating 
capital.  For  labor,  the  ratio  of  resources  to  output  is 
calculated  by  dividing  the  labor  cost  by  the  total  value  of 
the  production  to  obtain  "required  labor  cost  per  dollar  of 
production  value."  [Ref.  8:  p.  38]  For  materials,  the  ratio 
of  resources  to  output  is  found  by  dividing  cost  of 
materials  by  the  value  of  production  during  a  given  period 
of  time.  For  equipment  the  resources/output  ratio  is 
determined  by  dividing  the  annualized  value  of  the  equipment 
by  the  total  value  of  annual  production.  The  operating 
capital/output  ratio  is  similarly  the  average  monthly  value 
of  capital  divided  by  the  average  monthly  value  of 
production.  [Ref.  8:  p.  38] 

In  addition  to  efficiency  and  the  ratio  between 
resources  and  outputs,  there  are  special  criteria  for  each 
major  resource.  Some  of  the  special  criteria  for  labor. 
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materials,  equipment,  and  operating  capital  resources  are: 

•  Labor:  percent  of  employees  capable  of  filling  more 
than  one  position,  employee  turnover,  absenteeism, 
training  of  new  employees . 

•  Materials:  percent  of  work  in  process,  final 
products,  raw  materials,  percent  of  material 
rejected,  wasted  material. 

•  Equipment:  set  up  time,  down  time,  maintenance  cost 
per  period  of  time. 

•  Operating  capital:  percentage  of  various  components 
of  operating  capital.  [Ref.  8:  p.  38] 


3.  Brancato's  Performance  Criteria  Categories 

Brancato  [Ref.  6]  presents  five  measurement  categories 
that  reflect  her  research  on  a  number  of  companies 
throughout  the  world.  The  five  measurement  criteria 
include:  customer  satisfaction,  workplace  practices, 

relationships  with  suppliers,  environmental  competitiveness 
and  safety,  and  innovation.  Each  is  briefly  discussed 
below. 


a.  Customer  Satisfaction 

Companies  realize  that  it  is  more  profitable  to 

retain  an  existing  customer  than  to  recruit  a  new  one. 

Therefore,  there  is  a  definite  financial  benefit  to  customer 

satisfaction.  Research  shows  that: 

Seventy  percent  of  the  time,  a  very  dissatisfied 
customer  will  never  buy  from  you  again.  A  very 
satisfied  customer  will  buy  from  you  again  85 
percent  of  the  time.  In  terms  of  what  it  costs  to 
go  out  and  get  a  new  customer,  I  think  the 
research  indicates  that  it's  about  five  times  as 


32 


expensive  to  get  back  a  customer  than  to  keep  one. 

[Ref.  6:  p.  23] 

Customer  retention  is  also  a  key  profitability  forecasting 
tool.  As  the  length  of  the  customer- supplier  relationship 
increases,  there  is  an  increase  in  the  number  of  purchases; 
there  is  also  an  increase  in  the  "cross-cell"  ratio  as 
customers  purchase  other  related  products.  It  is  important 
to  measure  customer  satisfaction  with  product  quality, 
service  quality  and  cycle  times,  at  a  minimum.  [Ref.  6:  p. 
23] 

b.  Workplace  Practices 

According  to  Brancato,  most  companies  believe  that 
workplace  practices  should  interact  with  both  the 
performance  measurement  process  and  the  strategic  vision. 
This  type  of  intellectual  capital  can  be  a  company's  most 
important  asset,  one  having  hidden  value.  Brancato  defines 
intellectual  capital  as  "...the  result  of  the  linkage  of 
human  capital  (training  and  the  accumulation  of  knowledge, 
skills,  and  experience)  with  organizational  processes  in  the 
company  to  translate  that  human  capital  into  customer 
satisfaction,  high  quality  output,  productivity,  and, 
ultimately,  into  improved  financial  performance."  [Ref.  6: 
p.  27]  She  goes  on  by  saying: 

Some  companies  can  be  undervalued,  because  they 
possess  considerable  hidden  values  that  are  not 
accounted  for  in  the  book  value . . . these  hidden 
values  differentiate  companies  and  give  them  a 
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competitive  edge.  Therefore,  grasping  and 
systematically  managing  the  resources  that 
contribute  to  intellectual  capital  is  essential. 

[Ref.  6:  p.  27] 

Basic  workplace  practices  such  as  training,  team  processes, 
employment  security,  and  employee  satisfaction  should  lead 
to  improved  customer  satisfaction  (intermediate  measures) . 
This  ultimately  improves  profits  (final  measures).  [Ref.  6: 
p.  27] 

c.  Relationships  With  Suppliers 

Closer  working  relations  between  customers  and 
suppliers  can  reduce  costs,  improve  manufacturing 
efficiency,  reduce  inventories  and  improve  quality, 
ultimately  benefitting  both  customer  and  supplier.  [Ref.  6: 
p.  32] 

d.  Environmental  Competitiveness  and  Safety 

The  pressure  of  complying  with  ever-increasing 
environmental  regulations  makes  it  increasingly  important  to 
measure  and  track  environmental  performance .  The  key  to 
success  is  being  proactive  instead  of  reactive,  and  reduce 
pollutants  before  the  production  process.  This  reduces  the 
"ultimate"  pollution  before  it  is  discharged  into  the 
environment.  Frequently  it  is  cheaper  to  reduce  the 
pollutants  entering  the  production  process  than  to  remove 
the  pollutants  at  the  end  of  the  process.  This  practice 
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increases  quality,  improves  the  environment,  reduces  cost 
and,  ultimately,  increases  profitability.  [Ref.  6:  p.  32] 

e.  Innovation 

Many  companies  view  innovation  as  important  in 
achieving  competitive  success.  Innovation  was  traditionally 
measured  by  dividing  the  total  dollars  spent  on  research  and 
development  by  the  sales  or  profits.  Each  project  was  also 
evaluated  using  the  discounted  cash  flow  approach.  However, 
critics  point  out  that  this  method  "fails  to  link  the 
expenditures  to  viable  product  and  profitability  outputs" 
and  to  the  company's  strategic  vision.  [Ref.  6:  p.  33]  A 
recent  study  identifying  what  measures  successful  companies 
use,  shows  that  they  measure  innovation,  "...not  from  R&D 
dollars  spent,  but  from  the  percentage  of  sales  from  new 
products  introduced  within  the  last  specified  number  of 
years."  [Ref.  6:  p.  33]  Instead  of  using  the  traditional 
discounted  cash  flow  method,  these  successful  companies  base 
their  investment  decisions  on  the  likely  contribution  to  the 
company's  strategic  objectives.  [Ref.  6:  p.  33] 

4.  Kaplan  and  Norton's  Performance  Criteria 
Categories 

Based  on  a  year-long  research  project  involving  12 
companies  at  the  forefront  of  performance  measurement, 

Kaplan  and  Norton  [Ref.  9  and  10]  developed  the  "balanced 
scorecard."  The  scorecard  helps  managers  focus  on  a  handful 
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of  performance  criteria:  the  customer  perspective,  internal 
business  perspective,  innovation  and  learning  perspective, 
and  the  financial  perspective.  Like  the  other  authors, 
Kaplan  and  Norton  suggest  translating  measurement  criteria 
from  the  organization's  mission  statement.  Each  of  the  four 
measurement  criteria  presented  by  Kaplan  and  Norton  are 
summarized  below. 

a.  Customer  Perspective 

A  customer  focus  has  become  part  of  the  corporate 
mission  statement  for  many  companies.  Satisfying  the 
customer  has  become  a  top  priority  for  management.  Kaplan 
and  Norton  state  that  the  customer's  concerns  fall  into  four 
categories:  time,  quality,  performance  and  service,  and 

cost . 

Time,  or  lead  time,  measures  the  time  required  for 
the  company  to  fulfill  its  customers'  needs,  whether- it  be 
delivering  a  product  or  service.  For  new  products  or 
services,  lead  time  is  the  time  it  takes  to  develop  a 
product  or  service  from  the  product  definition  stage  to  the 
start  of  shipment  or  service  provision.  For  existing 
products,  lead  time  can  be  measured  from  the  time  the 
company  receives  the  order  to  the  time  the  service  or 
product  is  delivered.  [Ref.  9:  p.  73] 

Kaplan  and  Norton  measure  quality  as  the  number  of 
defects  per  product  as  perceived  by  the  customer.  Quality 
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can  also  measure  the  accuracy  of  the  company's  delivery 
claims  or  forecasts  by  measuring  the  on-time  delivery  rate. 
"The  combination  of  performance  and  service  measures  how  the 
company's  products  or  services  contribute  to  creating  wealth 
for  its  customers."  [Ref.  9:  p.  73] 

Cost  to  a  customer  is  more  than  just  price  per 
unit.  Customers  are  concerned  about  other  supplier  driven 
costs,  such  as:  schedule  disruptions  due  to  incorrect 
deliveries;  ordering  costs;  scrap,  rework,  and  obsolescence 
and  other  material  costs;  and  delivery  costs.  It  is  not 
uncommon  for  an  excellent  supplier  to  charge  a  higher  unit 
price  than  other  vendors.  The  high  price  supplier  may  be 
the  overall  low  cost  supplier  if  products  are  delivered 
defect -free,  to  the  right  place,  at  the  right  time,  and  in 
the  right  quantities.  [Ref.  9:  p.  74] 

For  the  balanced  scorecard  to  portray  the 
customers'  viewpoint,  it  is  important  for  the  organization 
to  define  its  goals  for  lead  time,  quality,  performance  and 
service,  and  cost,  then  translate  these  goals  into  specific 
measures.  Kaplan  and  Norton  find  that  companies 
implementing  this  approach  depend  on  the  customer's 
evaluations  to  define  some  of  the  company's  performance 
measures.  Involving  the  customer  to  define  lead  time, 
quality,  performance  and  service,  and  cost  has  helped 
companies  realize  that  customers  view  each  criteria 
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differently.  Those  companies  that  tailor  their  performance 
measures  to  their  customers  are  the  most  successful.  [Ref. 

9:  p.  73] 

ib.  Internal  Business  Perspective 

Superb  customer  service  is  derived  from  the 
processes,  actions,  and  decisions  made  throughout  an 
organization.  Therefore,  it  is  important  that  the  customer- 
based  measures  previously  described  be  translated  into 
measures  of  what  the  company  must  do  internally  to  meet  its 
customer's  demands.  The  internal  business  aspect  of  the 
balanced  scorecard  must  focus  on  the  internal  processes 
which  have  the  greatest  effect  on  customer  satisfaction.  It 
is  also  important  for  organizations  to  identify  their  core 
competencies  (what  they  do  best) .  Core  competencies,  along 
with  the  critical  success  factors  including  cycle  time, 
quality,  employee  skills,  and  productivity,  must  be  measured 
to  enable  the  company  to  improve  over  time.  To  improve 
cycle  time,  quality,  productivity,  and  cost,  managers  must 
have  full  employee  involvement  in  devising  the  measurement 
system.  This  helps  ensure  acceptance  and  understanding,  and 
will  contribute  to  success.  [Ref.  9:  p.  74] 

c.  Innovation  and  Learning  Perspective 

In  today's  global  competitive  business 
environment,  businesses  are  required  to  continually  improve 
their  existing  products  and  release  new  products.  Customers 
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are  always  looking  for  new  products  with  expanded 
capabilities  that  add  more  value.  In  order  for  a  company  to 
survive  and  grow  by  penetrating  new  markets,  it  is  important 
that  the  company  continue  to  release  new  products,  improve 
their  operating  efficiencies  and  increase  the  value  for 
their  customers.  In  other  words,  a  company's  value  is 
directly  tied  to  the  company's  ability  to  innovate,  improve 
and  to  learn.  Many  companies  measure  percent  of  sales  from 
new  products  to  reflect  success  in  innovation.  [Ref.  9:  p. 
75] 

d.  Financial  Perspective 

Critics  of  financial  measures  argue  that 
competition  has  changed  recently  and  the  traditional 
financial  measures  do  not  improve  customer  satisfaction, 
quality,  cycle  time,  or  employee  motivation.  They  state 
that  a  company' s  financial  performance  is  a  result  of 
fundamental  operational  actions.  As  long  as  companies 
perform  these  fundamental  operations,  the  financial  measures 
will  follow.  They  also  argue  that  traditional  financial 
measures  are  backward  looking;  they  do  not  measure  the 
organization's  ability  to  create  future  value. 

Kaplan  and  Norton,  however,  argue  that  financial 
measures  are  still  valid.  They  state  that  "a  well  designed 
financial  control  system  can  actually  enhance  rather  than 
inhibit  an  organization's  total  quality  management  program." 
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[Ref.  9:  p.  77]  More  important,  Kaplan  and  Norton  argue 
that,  an  improved  operating  performance  does  not  guarantee 
improved  financial  performance.  An  organization  may  improve 
quality  or  on-time  delivery  performance,  and  yet  not 
'-^pitalize  on  these  operational  achievements.  Thus,  it  may 
fail  at  improving  financial  performance. 

ip  is  important  that  organizations  follow  up  their 
operational  improvements  with  another  round  of  actions. 

Thus,  financial  performance  criteria  are  still  important. 

The  goals  of  financial  performance  measures  are  to  ensure 
the  company's  strategy,  implementation,  and  execution  are 
contributing  to  profitability,  growth,  and  shareholder 
^slue .  Finally  it  is  important  that  companies  specify  how 
improvements  made  in  quality,  cycle  time,  lead  times. 
Productivity,  and  new  products  will  improve  the  company's 
market  share,  operating  margins,  and  asset  turnover  or 
reduce  its  operating  expenses.  [Ref.  9:  p.  77] 

F .  SUMMARY 

This  literature  review  builds  a  foundation  for 
performance  measures:  what  they  are,  why  they  are  important, 
the  need  for  new  measures,  along  with  various  views  on 
measurement  criteria  design.  There  are  some  commonalities 
between  all  the  proposals  as  well  as  some  differences.  What 
is  important,  however,  is  that  no  author  has  insisted  that 
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their  set  of  criteria  is  the  only  correct  set.  Typically, 
authors  emphasize  that  each  organization  determine  what  set 
is  appropriate  for  their  line  of  business  and,  most 
importantly,  get  employees,  customers  and  suppliers  involved 
in  determining  the  individual  measurement  criteria  and 
attributes.  This  approach  best  assures  that  the  measurement 
criteria  are  accepted  and  understood,  enhancing  the 
prospects  for  overall  program  success. 

Most  authors  generally  agree  that  the  measures  should 
be  linked  to  the  organization's  goals,  objectives  and 
strategies.  For  maximum  success,  it  is  best  to  start  with  a 
clean  sheet  of  paper.  After  developing  proposed  measures, 
the  new  list  can  be  compared  to  existing  measures  to  fill  in 
any  holes.  Section  B  discussed  some  of  the  roles 
measurement  can  play;  such  as :  ensuring  strategy  is 
implemented,  a  management  support  system,  a  control  device, 
and  for  improvement.  Most  authors  considered  measurement 
for  improvement  to  be  the  most  important  role.  This  thesis 
will  be  based  on  that  premise. 

Finally,  Table  1  provides  an  overview  of  the 
measurement  criteria  categories  presented  by  the  authors 
discussed  in  this  chapter.  The  last  column  lists  the 
general  measurement  categories  used  for  MSC  managed  Special 
Mission  oceanographic  ships  in  Chapter  IV  of  this  thesis. 
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III.  NAVOCEANO  AND  CURRENT  METRICS 


This  chapter  provides  background  on  the  Naval 
Oceanographic  Office  (NAVOCEANO),  one  of  MSC's  major  Special 
Mission  Ship  sponsors.  The  first  section  will  describe 
NAVOCEANO' s  mission,  the  ships  they  use  and  their  operating 
characteristics.  The  following  section  describes  the 
services  MSC  provides  to  NAVOCEANO.  A  list  of  measures  that 
MSCPAC  and  MSCLANT  are  currently  collecting  or  tracking  is 
listed  next.  Finally,  various  issues  which  NAVOCEANO  and 
the  General  Accounting  Office  (GAO)  have  with  MSC's  service 
are  discussed. 

A.  NAVAL  OCEANOGRAPHIC  OFFICE 

1 .  Background 

Naval  Oceanographic  Office  (NAVOCEANO) ,  located  at 
Stennis  Space  Center,  Bay  St.  Louis,  Mississippi,  employs 
approximately  1,000  military  and  civilian  personnel  to 
support  oceanographic  requirements  worldwide.  NAVOCEANO  is 
the  largest  component  of  its  parent,  COMNAVMETOCCOM 
(Commander,  Naval  Meteorology  and  Oceanography  Command) . 
"COMNAVMETOCCOM  is  tasked  with  executing  the  Navy 
Oceanography  Program  and  fulfilling  validated  requirements, 
and  provides,  as  major  claimant,  the  necessary  OM&N  funds  to 
operate  and  maintain  the  Special  Mission  Oceanographic 
Fleet."  [Ref.  11]  NAVOCEANO' s  mission  is  "to  provide 
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specialized  and  unique  oceanographic  products  and  services 
to  joint  warfighters  in  a  manner  and  timeframe  that  allows 
them  to  meet  their  objectives."  [Ref.  12]  The  Navy's  Fleet 
Commanders  request  surveys  from  the  Defense  Mapping  Agency 
(DMA) .  DMA  tasks  NAVOCEANO,  through  COMNAVMETOCCOM,  to 
perform  those  surveys.  [Ref.  13]  NAVOCEANO  collects 
hydrographic,  magnetic,  geodetic,  chemical,  navigation  and 
acoustic  data  using  a  variety  of  platforms  including  ships, 
aircraft  and  spacecraft.  In  recent  years,  NAVOCEANO' s  focus 
has  changed  from  deepwater  survey  operations  to  coastal  or 
littoral  regions,  due  to  our  changing  defense  strategy  with 
the  end  of  the  Cold  War.  [Ref.  14] 

NAVOCEANO  performs  two  types  of  surveys :  hydrographic 
and  oceanographic . 

Hydrographic  surveys  are  conducted  to  measure  and 
describe  the  physical  features  of  the  ocean.  The 
information  is  required  to  ensure  the  safe 
navigation  of  all  United  States'  ships  outside  our 
national  territorial  waters.  The  data  is  provided 
to  the  Defense  Mapping  Agency  (DMA)  for  processing 
and  printing  of  nautical  charts.  [Ref.  14] 

The  charts  are  then  distributed  to  the  Fleet  Commanders . 

Hydrographic  surveys  are  usually  done  in  the  territorial 

waters  (12  NM)  of  another  country  with  their  permission.  In 

which  case,  the  United  States  agrees  to  share  all  the  survey 

data  with  that  host  country  and  extends  an  invitation  for 

one  person  from  that  country  to  participate  in  the  survey. 

Hydrographic  surveys  are  usually  unclassified.  [Ref.  13] 
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The  second  type  of  survey  is  the  oceanographic  survey. 
This  encompasses  the  whole  water  column  down  to,  and 
including,  the  bottom.  Oceanographic  surveys  measure  such 
things  as  depth  of  water,  temperature,  salinity,  sound 
velocity,  earth's  magnetic  field  variations  and  gravity 
anomalies.  They  also  take  bottom  core  samples  to  help 
understand  more  about  the  shape  and  the  texture  of  the  ocean 
floor.  Oceanographic  surveys  are  rarely  done  in  another 
country's  territorial  waters,  but  are  usually  within  their 
Exclusive  Economic  Zone  (EEZ) ,  200  NM  out  frbm  shore.  In 
this  case,  they  are  termed  "Military  Surveys, "  to  legally 
avoid  having  to  get  permission  from  the  host  country.  If 
U.S.  ships  are  conducting  "research,"  we  could  possibly  find 
oil,  manganese  nodules  or  fisheries  data.  This  information 
could  impact  the  economic  well-being  of  the  host  country; 
they  would  want  to  be  fully  informed.  A  military  survey 
does  not  economically  threaten  a  host  country  because  the 
data  is  intended  for  war-fighting  not  economic  purposes. 
Oceanographic  surveys  can  either  be  classified  or 
unclassified.  [Ref.  13] 

Surveys  in  recent  years  have  shifted  to  littoral 
regions  in  the  Western  Pacific,  Persian  Gulf,  and  the 
Eastern  Mediterranean  Sea,  as  well  as  other  areas  that  have 
high  regional  military  interest.  Requests  from  prior 
Eastern  Bloc  countries  to  jointly  survey  territorial  waters 
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are  more  common  in  recent  years.  This  data  has  been  a 
benefit  to  both  the  Eastern  Bloc  country  and  the  United 
States .  [Ref .  13] 

2 .  NAVOCEANO  Ships  Managed  by  MSC 

NAVOCEANO  currently  has  five  ships  that  are  in  full 
operating  status  (FOS)  and  are  conducting  surveys.  The 
older  ships,  originally  designed  to  accomplish  a  specific 
mission,  are  continually  being  modified  for  multi-purpose 
operations,  providing  more  flexibility.  The  new  ships  are 
designed  for  multi-purpose  operations,  including  shallow 
water  surveying  in  littoral  areas. 

The  USNS  SILAS  BENT  and  USNS  KANE  are  30  year-old  ships 
of  the  SILAS  BENT  class.  They  are  approximately  285  feet  in 
length.  They  perform  oceanographic  surveys  in  the  EEZ,  but 
outside  the  territorial  waters.  They  are  equipped  with 
large  winches  for  drawing  bottom  samples.  The  USNS  WYMAN  is 
a  member  of  the  SILAS  BENT  class,  but  was  never  equipped 
with  large  winches.  It  isn't  capable  of  deep-sea  coring  or 
towing  transducers.  It  performs  oceanographic  surveys 
outside  the  12  nautical  mile  territorial  waters  and  was 
designed  primarily  for  deep  ocean  bathymetry. 

The  USNS  LITTLEHALES  and  the  USNS  JOHN  MCDONNELL  are 
approximately  4  years  old  and  208  feet  in  length.  They  have 
two  sounding  boats  which  can  be  deployed  to  perform  bottom 
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mapping.  These  ships  primarily  conduct  shallow  water 
hydrographic  surveys  within  territorial  waters . 

NAVOCEANO  owns  two  additional  ships  which  are  in  FOS, 
but  are  not  currently  conducting  survey  work.  The  USNS 
PATHFINDER  and  the  USNS  SUMNER  are  members  of  the  new  T-AGS 
60  class  of  ships  which  are  329  feet  in  length  and  are 
considered  true  multi-purpose  oceanographic  ships.  They  are 
new  construction  and  are  currently  undergoing  acceptance 
trials  before  being  assigned  to  active  survey  work  in  mid  to 
late  1996 .  The  MSC  per  diem  charges  are  the  same  whether  an 
FOS  ship  is  in  port  or  at  sea. 

Three  ships  are  currently  in  reduced  operating  status 
(ROS) .  The  BOWDITCH  and  the  HENSON  are  members  of  the  T-AGS 
60  class;  they  will  be  delivered  to  the  Navy  in  mid  1996  and 
1998  respectively,  and  are  capable  of  multi-purpose 
operations.  As  with  the  PATHFINDER  and  SUMNER,  they  are 
designed  for  deep  ocean  surveys,  oceanographic  surveys  and 
shallow  water  survey  operations  inside  territorial  waters 
using  sounding  boats.  They  also  have  large  deck  winches  to 
collect  bottom  core  samples. 

The  third  ROS  ship,  the  USNS  WATERS,  has  only  a 
skeleton  MSC  civilian  mariner  crew.  This  4  year  old  ship 
was  formerly  used  by  SPAWARs  as  a  cable  laying  ship  .  It 
will  be  modified  for  multi-purpose  survey  work  under 
NAVOCEANO.  One  additional  ship,  the  T-AGS  64,  which  is  yet 
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unnamed,  is  awaiting  construction  funding  from  Congress  and 
is  scheduled  to  be  delivered  1  Nov  2000. 

The  size  of  the  mariner  crews  range  from  28  to  31  for 
the  older  ships;  crew  size  is  22  for  the  T-AGS  60  class.  In 
addition,  each  ship  carries  a  crew  of  about  12  NAVOCEANO 
personnel,  civilian  and  military,  to  perform  the  survey 
work.  All  the  active  ships  in  FOS  status  are  currently 
contractor  operated;  the  PATHFINDER  and  the  SUMNER  are 
manned  by  MSC  civilian  mariners  (CIVMARS) .  All  the  ships  in 
ROS  are  manned  by  civilian  mariners.  The  DYNCORP  contract 
currently  allows  up  to  six  ships  to  be  contractor  operated. 
Additional  oceanographic  ships  must  be  civilian  manned  or 
operated . 

In  November  1995,  Admiral  Boorda  released  a  new 
Oceanography  Policy  statement.  It  requires  a  minimum  of 
eight  active  oceanographic  ships.  [Ref.  15:  p.  38-44]  As 
the  newer  ships  become  available  for  active  survey  work,  the 
SILAS  BENT  class  will  be  retired  to  maintain  eight 
oceanographic  ships.  [Ref.  13]  [Ref.  16:  p.  5] 

3.  NAVOCEANO' S  Ship  Operations 

NAVOCEANO  ships  average  250  days  at  sea  and  115  days  in 
port,  either  at  a  port  of  call,  in  upkeep  or  in  overhaul. 

The  typical  schedule  is  25  days  at  sea  and  four  days  in 
port.  Port  calls  usually  involve  a  personnel  changeout, 
equipment  changeout  and  maybe  a  short  upkeep.  Unlike  naval 
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warships,  survey  ships  have  no  specific  home  port.  They 
have  recently  been  allowed  to  undergo  their  annual  overhaul 
in  foreign  shipyards,  which  will  drastically  reduce  the 
costs . 

NAVOCEANO's  active  survey  ships  are  manned  by  contract 
mariners  from  DYNCORP.  They  typically  sign  on  for  four  to 
six  months  at  a  time.  The  NAVOCEANO  survey  personnel,  which 
include  civilian  and  military,  are  usually  rotated  every  60 
days .  Both  the  survey  personnel  and  contract  mariners  are 
rotated  in  a  staggered  manner. 

Surveys  are  done  primarily  in  the  Western  Pacific, 
Persian  Gulf,  and  the  Eastern  Mediterranean.  Recently,  some 
ships  have  been  diverted  to  areas  that  are  of  high  current 
military  interest  in  order  to  collect  survey  data  for  use  by 
the  Department  of  Defense  and,  if  in  territorial  waters, 
shared  with  the  host  country.  Some  former  Eastern  Bloc 
countries  have  requested  survey  work  within  their 
territorial  waters.  These  are  given  high  priority  and 
require  diplomatic  clearances.  [Ref.  13] 

NAVOCEANO  publishes  an  annual  schedule  for  its  survey 
ships.  This  schedule  is  updated  quarterly.  There  is  also  a 
five  year  plan.  This  is  formulated  by  NAVOCEANO, 
incorporating  tasking  from  DMA  which  is  communicated  through 
COMNAVMETOCCOM .  Once  the  annual  schedule  has  been  released, 
few  changes  occur.  Any  major  schedule  changes  are  usually 
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identified  at  least  a  year  in  advance.  Changes  in  the 
quarterly  updates  are  usually  minor,  primarily  affecting 
port  calls. 

Schedules  also  consider  the  diplomatic  clearance 
processing  time  required  for  surveying  in  foreign 
territorial  waters.  These  usually  require  a  long  lead  time. 
As  a  condition  for  obtaining  a  clearance,  the  U.S.  agrees  to 
share  the  survey  data  with  the  host  country  and  typically 
invites  one  or  two  hydrographers  or  oceanographers  to  assist 
NAVOCEANO  in  collecting  the  survey  data.  Clearances  are 
only  good  for  a  certain  window  of  time.  Therefore,  it  is 
important  to  accurately  schedule  those  operations. 

Equipment  change  outs  are  accomplished  within  5  days  in 
the  nearest  port.  They  don't  require  a  shipyard.  The  ships 
call  on  the  nearest  port  which  can  provide  the  services, 
with  an  emphasis  on  minimizing  the  time  and  distance  to  a 
port  and  port  call  time  to  get  the  ship  back  out  on  station 
as  quickly  as  possible.  The  contractor  will  send 
representatives  or  agents  to  those  ports  of  call  to  arrange 
for  required  services.  [Ref.  13] 

Ships  may  revert  to  an  ROS  status  during  overhauls  or 
occasionally  when  a  C-4  CASREP  (serious  equipment 
malfunction  which  prevents  the  ship  from  accomplishing  its 
mission)  prevents  them  from  getting  underway  for  a  period 
greater  than  one  month.  During  ROS,  the  ship's 
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watchstanders  are  reduced  to  the  minimum  required  number. 
That  reduces  the  per  diem  rate.  [Ref.  13] 

B.  WHAT  MSC  PROVIDES  FOR  NAVOCEANO 

A  memorandum  of  agreement  (MOA)  between  the  Commander, 
Military  Sealift  Command  (COMSC)  and  the  Commander,  Naval 
Meteorology  and  Oceanography  Command  (COMNAVMETOCCOM)  lays 
out  the  responsibilities  of  each  party  in  managing  the 
oceanographic  ships.  COMSC  is  typically  responsible  for 
support,  administrative  control  and  operational  control  of 
the  ships.  COMSC  delegates  these  responsibilities  to  the 
appropriate  COMSC  area  commanders .  COMNAVMETOCCOM  has 
technical  control  of  the  ships  and  provides  funding  support 
for  the  ship's  operation  and  maintenance.  COMNAVMETOCCOM 
delegates  ship  technical  control  to  the  Naval  Oceanographic 
Office  (NAVOCEANO).  [Ref.  17:  p.  2] 

The  following  is  a  list  of  some  of  the  services  that 
MSC  provides  NAVOCEANO: 

•  Operational  oversight,  including  scheduling  and 
activation/deactivation  planning,  through  MSC  Area 
Commanders 

•  Maintenance  and  repair  including  configuration 
control  and  integrated  logistics  support 

•  Contract  management  and  quality  assurance  of  contract 
operators 

•  Finance  and  accounting  services  including  budget/POM 
submittals  and  invoice  certification  and  payment 
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•  Maintenance  of  insurance  reserves  for  accidents  and 
claims 

•  Expertise  in  admiralty  law,  contract  law,  maritime 
contract  law,  maritime  insurance  law,  environmental 
law  and  engineering  repair 

•  Independent  compliance  checks  for  US  Coast  Guard 
(USCG) /American  Bureau  of  Shipping  (ABS) 
certification 

•  Maintenance  of  ISO  9000  quality  standards 

•  Compliance  with  Navy  mandated  pollution  control 
requirements  such  as  oil  pollution  and  spill 
response,  HAZMAT  control,  plastics  at  sea,  ozone 
depleting  substances,  and  NAVOSH 

•  Personnel  and  training  management.  [Ref.  18]  [Ref 
19] 


C.  MEASURES  CURRENTLY  TRACKED  BY  MSC 

The  research  methodology  includes  comparing  proposed. 
MSC  performance  measures  developed  independently  in  Chapter 
IV  with  those  currently  being  tracked  by  MSCPAC  and  MSCLANT 
Special  Missions  personnel.  This  comparison  will  reveal 
various  holes  and  overlap  between  the  two  lists . 

The  following  is  a  list  of  performance  measures 
provided  by  MSCPAC.  These  are  generic  and  not  specific  to 
the  Special  Mission  Ship  program  because  MSCPAC  currently 
does  not  manage  any  oceanographic  ships . 

•  Status  Of  Resources  and  Training  System  (SORTS)  data 

•  Equipment  Casualty  Reporting  (CASREP)  system  data 

•  Number  of  days  mariner  waits  in  the  pool 

•  Training  pipeline  costs 
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•  Work  progress  of  ships  in  maintenance  availability 

•  Daily  ship  performance  through  operational  reports 
(MOVREPS)  and  daily  summaries 

•  Number  of  days/yr  ship  is  available  to  perform 
mission 

•  Cost  per  available  day 

•  Billing  data  [Ref.  20] 

MSCLANT  (Special  Mission  Ships  East)  provided  a  more 
comprehensive  list  of  measures  broken  down  by  contract 
operated  ships  and  civil  service  manned  ships .  The  more 
detailed  list  reflects  that  MSCLANT  is  the  Contracting 
Officer  Technical  Representative  (COR)  for  the  DYNCORP 
oceanographic  ship  contract.  The  following  measures  are 
tracked.  Some  are  controllable  and  are  actually  managed 
while  some  are  just  indicators  for  monitoring: 

CONTRACT  OPERATED  SHIPS 

•  Actual  survey  time  versus  sponsor  requested 
operations 

•  External  audits 

•  Control  discrepancy  report  tracking 

•  Contractor  Invoice  Review  and  Certification 

•  Overall  maintenance  and  repair  oversight 

•  Port  engineer  attendance  at  overhauls 

•  Review  of  specification  packages 

•  Administrative  contracting  officer  approval  of  work 
and  costs 

•  Validated  user  complaints  from  sponsor 
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•  On-hire/of f -hire  management 

•  ROS/FOS/RAV  status  control  (Reduced  Operational 
Status,  Full  Operational  Status,  and  Repair 
Availability) 

•  SORTS,  CASREPS,  Movement  Report  (MOVREP) 

•  Weekly  Operational  Summary  Reports  (WOSR)  or  Regular 
Overhaul  reports  (ROH)  [Ref.  21] 

CIVIL  SERVICE  MANNED  SHIPS 

•  Actual  survey  time  versus  sponsor  time  requested 

•  CIVMAR  performance/assignments 

•  SORTS  reporting 

•  CASREP  reporting 

•  MOVREP  reporting 

•  WOSR/ROH  reports 

•  ROS/FOS  management 

•  Per  diem  structure 

•  Customer  satisfaction 

•  Training/pipeline  costs  [Ref.  21] 

The  Special  Mission  Ships  Program  Manager  was  unable  to 
provide  a  list  of  measures  tracked  by  COMSC.  However, 
measures  are  a  high  priority.  Within  the  next  six  months, 
COMSC  will  be  developing  their  own  measures  in  conjunction 
with  their  reinvention  efforts. 
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D.  ISSUES  NAVOCEANO  AND  GAO  HAVE  WITH  MSC  SERVICE 

Various  issues  surfaced  when  interviewing  Operations 
and  Financial  directorate  personnel  at  NAVOCEANO  and 
COMNAVMETOCCOM  between  29  JAN  96  and  1  FEB  96.  Additional 
issues  were  identified  in  the  newly  released  General 
Accounting  Office  (GAO)  report  entitled  Military  Sealift 
Command:  Weak  Controls  and  Management  of  Contractor- 

Operated  Ships.  The  following  discussion  of  issues  is  not 
meant  to  be  an  all  inclusive  list  but  represents  some  key 
concerns  of  MSC's  customers  at  one  point  in  time.  These 
concerns  were  used  in  constructing  performance  measures  for 
MSC's  Special  Mission  Ships. 

Each  issue  will  be  discussed  briefly  below.  This  list 
is  not  in  any  specific  order  of  priority  or  importance. 

The  discussion  will  not  recommend  particular  solutions  to 
each  issue.  Instead,  the  following  chapter  will  propose 
measures  with  which  MSC  can  gauge  its  improvement  in  serving 
its  customers. 

1 .  NAVOCEANO  Issues 

a.  Maintenance  of  Mission  Related  Deck  Equipment 

(MRDE) 

Properly  operating  and  maintaining  all  mission 
related  deck  equipment  (MRDE)  is  critical  for  completing  the 
oceanographic  ship's  mission.  MRDE  is  so  important  to 
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NAVOCEANO,  that  its  proper  maintenance  is  spelled  out  both 
in  the  MOA  between  COMSC  and  COMNAVMETOCCOM  [Ref.  17:  p.  7] 
and  the  contract  between  COMSC  and  DYNCORP  [Ref.  22:  p.  73- 
74]  . 

For  example,  when  MSC  and  NAVOCEANO  drew  up  the 
contract  statement  of  work  (SOW)  requirements,  they 
stipulated  that  the  contractor  must  provide  one  Port 
Engineer  who  should  have  the  ability  "demonstrated  through 
training  and  experience,  to  manage  the  maintenance, 
troubleshooting  and  repair  of  mission  related  deck 
machinery,  hydraulic  systems  and  related  electrical 
systems."  [Ref.  22:  p.  19]  The  contractor  must  also  provide 
a  monthly  mission  related  deck  equipment  status  report, 
summarizing  the  condition  of  all  the  mission  related  deck 
equipment  [Ref.  22:  p.  74]. 

During  its  annual  ship  check,  NAVOCEANO  is 
discovering  signs  of  neglect  in  both  maintaining  MRDE  and 
reporting  its  condition  despite  the  emphasis  they  receive 
both  in  the  MOA  and  in  the  contract.  Specifically,  some  of 
the  shortcomings  identified  in  January  1996  were  dry, 
unlubricated  gears  inside  gear  boxes;  winches  that  could  not 
be  tested  because  the  fairlead  blocks  had  frozen;  winches 
that  had  been  out  of  service  for  several  months  despite  a 
monthly  status  report  submitted  just  five  days  earlier 
stating  all  equipment  was  100%  ready  (the  NAVOCEANO  Sponsor 
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Designated  Representative  (SDR)  concurred  with  that  report) . 
It  is  not  Jcnown  whether  these  discrepancies  have  yet 
impacted  the  ship's  mission.  [Ref.  23] 

b.  Ship  Configuration  Management 

The  MOA  states  "MSC  shall  have  direct 
responsibility  for  authority  over  material  condition  for 
areas  assigned  in  Appendices  II  and  III."  [Ref.  17:  p.  9] 

The  shipboard  automated  maintenance  management  (SAMM)  system 
"was  developed  to  provide  an  accurate  and  efficient  means  to 
schedule,  document,  track,  report,  and  manage  shipboard 
maintenance.  The  SAMM  system  also  maintains  accurate 
machinery  history  and  equipment  configuration."  [Ref.  22:  p. 
297]  SAMM  includes  a  data  base  of  all  the  ship's  equipment 
broken  down  by  components  with  narratives  describing  the 
proper  maintenance  for  these  components.  This  system  is 
similar  to  the  active  Navy' s  Material  Maintenance  Management 
(3M)  system.  However,  advocates  say  SAMM  is  much  easier  to 
manage . 

MSC,  through  it's  contractor  DYNCORP,  is 
responsible  for  updating  SAMM  each  time  there  is  an 
equipment  deletion  or  addition.  Updates  to  the  ship's 
configuration  file  are  submitted  by  the  contractor  through 
COMSC.  MSCLANT  and  COMSC  Engineering  personnel  review  those 
configuration  changes  and  forward  them  to  Seaworthy,  the 
contractor  who  designed  and  implemented  the  SAMM  system. 
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Seaworthy,  in  turn,  mails  out  ship  configuration  file 
updates  via  disk  to  all  the  ships. 

NAVOCEANO ' s  annual  ship  checks  reveal  that 
additions  and  deletions  to  the  MRDE  are  not  updated  in  the 
ship's  configuration  file.  Specifically,  the  ship  checks  in 
January  1996  of  the  KANE,  WYMAN  and  LITTLEHALES  revealed 
that  some  of  the  MRDE  equipment  and  many  of  the  components 
are  not  listed.  Some  of  the  maintenance  action  narratives 
match  the  equipment  in  name  only  and  the  configuration  files 
do  match  the  name  tag  data  on  the  equipment . 

The  result  is  incomplete  files  of  equipment 
actually  on  the  ship,  missing  or  improper  preventive 
maintenance  (PM)  and  corrective  maintenance  (CM)  narratives 
for  the  components,  inadequate  tech  manual  support,  and 
inadequate  Coordinated  Shipboard  Allowance  List  (COSAL) 
support  for  the  equipment .  This  ultimately  leads  to 
inadequate  repair  part  support  and  degraded  material 
readiness.  [Ref.  23] 

c.  MSC  Overhead  Costs 

NAVOCEANO,  along  with  other  Special  Missions 
Program  sponsors,  are  very  concerned  about  the  overhead 
costs  paid  to  MSC  for  managing  their  ships.  They  suspect 
that  MSC's  overhead  costs  are  excessive  (as  evidenced  by 
their  desire  to  obtain  cost  estimates  from  outside  ship 
management  firms)  [Ref.  11] .  They  believe  they're  paying 


more  than  their  fair  share  for  what  they  receive  back  in 
services.  MSC  hopes  that  reinventing  and  adopting  a  Program 
Manager  organization  structure  will  greatly  reduce  its 
overhead  costs.  (To  date,  COMSC  has  not  released  any 
estimates  of  its  expected  savings  from  changing  to  a  program 
management  structure.)  Unless  MSC  can  impress  ship  sponsors 
by  improving  service,  sponsors  will  continue  to  look  for 
alternatives.  [Ref.  11] 

d.  MSC's  Management  of  Sponsor  Funds 

In  this  era  of  declining  military  budgets,  it  is 
important  that  MSC  be  accountable  and  responsible  for  the 
sponsors'  funds  which  pay  them.  In  March  and  April  1995, 

MSC  returned  several  balances  to  COMNAVMETOCCOM  from  prior 
year  work  orders .  These  funds  were  returned  too  late  to  be 
used  for  any  other  purpose .  COMNAVMETOCCOM  received 
$1,367,762  of  fiscal  Year  1991  through  1993  money,  which  was 
recaptured  from  charter  and  hire  costs,  overhaul  and 
installations,  and  sponsor  reimbursables .  MSC  still  holds 
$831,430  in  contingency  for  legal  settlements.  It  should  be 
noted  that  MSC  has  since  been  working  with  NAVOCEANO  and 
COMNAVMETOCCOM  to  return  the  balances  in  a  more  timely 
manner  (i.e.,  at  the  end  of  overhaul,  upon  cancellation  of 
services  or  before  the  end  of  each  fiscal  quarter) .  [Ref. 

24] 
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e.  Overstated  Cost  Estimates 

Related  to  the  slow  return  of  the  sponsors'  money, 
there  is  also  a  problem  with  COMSC's  initial  cost  estimates 
for  such  items  as  overhaul,  installation  costs  and  sponsor 
reimbursables .  Of  the  expired  funds  available  for 
recapture,  45.9%  to  83.3%  of  the  original  authorized  amount 
was  returned  to  the  Sponsor.  The  percentage  still  held  in 
contingency  for  legal  settlement  ranged  from  37.1%  to  53.7%. 
This  suggests  that  MSC  has  taken  an  ultra-conservative 
position  in  holding  COMNAVMETOCCOM ' s  and  NAVOCEANO's  limited 
funds  for  contingencies.  [Ref.  24] 

f.  Ship  Sighting  Report 

The  contract  requires  the  contractor  to  submit  a 
Ship  Sighting  Report  via  Naval  Message  each  time  the  ship 
spots  a  foreign  ship  or  plane  (within  reason) .  [Ref.  22:  p. 
251]  NAVOCEANO  is  receiving  pressure  by  the  Office  of  Naval 
Intelligence  (ONI)  in  Suitland,  Maryland  for  not  submitting 
any  reports.  Due  to  their  unique  operating  areas,  some  of 
these  ships  are  known  to  be  sighting  non- friendly  foreign 
contacts.  However,  no  reports  are  being  transmitted.  This 
will  become  an  increasingly  important  issue,  now  and  into 
the  future,  as  NAVOCEANO  performs  surveys  inside  the 
territorial  waters  of  previous  Eastern  Bloc  countries  and 
other  areas  of  military  interest.  [Ref.  25] 
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Logistics 


Sr- 

The  MOA  delineates  logistics  responsibility  as 

follows : 

Upon  request,  MSC  shall  provide  or  arrange  for 
receipt,  temporary  storage,  packing,  crating,  and 
shipment  of  Sponsor's  material  at  remote  ports. 

The  sponsor  shall  provide  sufficient  advance 
notice  of  such  operations  to  enable  the  COMSC  Area 
Commander  to  provide  additional  support  through 
normal  procurement  action.  [Ref.  17:  p.  8] 

An  important  problem  area  for  NAVOCEANO  is  world 
wide  mail  delivery  to  the  ships.  Personnel  rely  on  the  mail 
service  for  personal  mail,  spare  parts,  high  priority 
requisitions,  official  correspondence  and  charts.  At  times, 
the  mail  may  not  get  delivered  due  to  Air  Mobility  Command 
delays  or  logistics  problems.  These  are  usually  beyond  the 
contractor's  capability  to  resolve.  Therefore,  it  is 
important  for  the  MSC  field  offices  to  work  with  the  Sponsor 
(SDR)  to  resolve  the  logistics  issue  and  minimize  the 
effects  on  the  ship's  mission.  [Ref.  11] 

h.  Quality  of  Life 

Food  quality  and  service  is  a  quality  of  life 
issue  that  NAVOCEANO  considers  important.  This  is  a 
critical  morale  factor  that  affects  both  NAVOCEANO' s  survey 
crew  and  the  contractor's  crew.  [Ref.  11] 
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2. 


GAO  Audit  Results 


Mr.  Jim  McDiarmid,  MSC's  new  Program  Manager  for  the 
Special  Missions  Program,  recommended  the  recently  completed 
GAO  audit,  entitled  Military  Sealift  Command:  Weak  Controls 
and  Management  of  Contractor -Operated  Ships.  He 
particularly  suggested  considering  the  report's 
discrepancies  and  recommendations  for  identifying 
performance  measures.  [Ref.  26]  The  following  discrepancies 
were  among  those  identified: 

•  Inadequate  documentation  and  review  of  crew-performed 
repairs  resulting  in  overpayments 

•  Insufficient  invoice  documentation  to  ensure  that 
subcontractor's  prices  are  fair  and  reasonable 

•  Inconsistent  requirements  for  documentation 
supporting  invoices 

•  MSC  field  staff  generally  not  involved  in  the  invoice 
review  process 

•  No  controls  to  prevent  contractors  from  circumventing 
requirements  to  receive  MSC's  prior  approval  for 
subcontracts 

•  No  consistent  verification  that  overhaul  work  is 
complete  and  prices  are  reasonable 

•  No  standard  procedures  to  develop  personnel 
requirements  including:  crew  qualification, 
citizenship,  security  clearance  and  trustworthiness 
evaluations 

•  No  systematic  approach  to  identify  and  implement  best 
practices 

•  Fragmented  lines  of  authority  impeding  sound 
management  controls  [Ref.  27] 

Correcting  the  above  items  can  improve  MSC's  customer 
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service,  especially  for  NAVOCEANO.  Unfortunately,  a 
measurement  system  alone  won't  correct  these  problems. 

Their  resolution  generally  requires  process  innovations. 

This  topic  is  beyond  the  scope  of  this  thesis.  Performance 
measures  will  simply  quantify  the  improvement  in  MSC's 
performance . 

The  trustworthiness  evaluation  is  important  to 
NAVOCEANO' s  sensitive,  mission .  It  determines  the  loyalty  of 
an  individual  by  conducting  "national  agency  checks  of 
Federal  Bureau  of  Investigation  and  other  agency  records  to 
divulge  negative  information  such  as  drug  and  alcohol  abuse 
and  felony  convictions."  [Ref.  27:  p.  35]  GAO  found  that 
the  trustworthiness  evaluations  were  being  processed  too 
slowly.  Of  those  processed,  more  than  10%  of  the  contractor 
crews  were  found  to  be  untrustworthy  and  were  subsequently 
removed  from  the  ships.  Of  the  four  oceanographic  ship  crew 
lists  which  GAO  checked  in  January  1995,  MSC  only  completed 
39  of  the  94  trustworthy  evaluations.  [Ref.  27:  p.  38] 

E.  SUMMARY 

The  purpose  of  this  chapter  was  to  discuss  NAVOCEANO, 
it's  mission  and  some  background  on  the  ships  that  MSC 
manages.  This  chapter  also  described  what  MSC  provides  in 
terms  of  ship  management  for  NAVOCEANO,  referencing  the  MOA 
and  the  contract .  The  performance  measures  currently  being 
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collected  and  tracked  by  MSCPAC  and  MSCLANT  were  presented, 
followed  by  some  of  the  shortcomings  in  MSC's  service 
identified  by  NAVOCEANO  and  GAO.  The  next  chapter  will 
propose  a  broad  set  of  measurement  categories,  along  with 
individual  metrics,  which  will  allow  MSC  to  monitor  ongoing 
improvements  in  the  Special  Mission  Oceanographic  Ships 
program. 
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IV.  PROPOSED  MEASURES 


This  chapter  proposes  a  set  of  measures  to  track 
improvements  in  MSC's  management  of  the  Special  Mission 
Oceanographic  Ships .  The  top  level  measurement  categories 
will  be  based  on  the  general  categories  synthesized  from  the 
literature  review  and  summarized  in  Table  1  of  Chapter  II. 
Drawing  on  interviews  with  Naval  Oceanographic  Office 
(NAVOCEANO)  personnel,  the  Memorandum  Of  Agreement  (MOA) , 
MSC's  contract  with  DYNCORP,  MSC's  Mission  and  Vision 
statement,  and  the  author's  personal  knowledge  and 
experience  with  U.S.  Naval  ship  operations,  some  lower  level 
measures  will  be  presented  which  could  apply  specifically  to 
the  MSC's  management  of  oceanographic  ships.  A  short 
discussion  will  follow  on  how  these  measures  can  help  solve 
some  of  the  current  issues  NAVOCEANO  and  GAO  have  identified 
with  regard  to  MSC's  ship  management. 

There  is  no  single  correct  way  for  structuring  the  top 
level  and  lower  level  measurement  hierarchy.  This  is  just  a 
proposed  framework.  The  final  decision  should  be  made  by 
those  involved  in  the  process,  including  MSC  personnel, 
customers,  suppliers,  etc.  All  participants  should  work 
together  to  identify  and  develop  measures  for  improvement . 
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A.  MEASURES  APPLICABLE  TO  OCEANOGRAPHIC  SHIP  MANAGEMENT 

Four  top  level  measurement  categories  have  been 
formulated  for  oceanographic  ship  management,  based  on  the 
general  measurement  categories  synthesized  from  the 
literature  review.  These  measures  are  summarized  in  Table  1 
of  Chapter  II.  They  include:  customer  satisfaction, 
internal  business  perspective,  innovation,  and  financial 
perspective.  For  simplicity  in  presenting  this  material, 
definitions  for  these  measures  are  provided  in  the  Appendix. 

1.  Customer  Satisfaction 

Customer  satisfaction  measures  focus  on  the  things 
which  are  most  important  to  NAVOCEANO ' in  fulfilling  its 
mission.  This  category  aggregates  four  sub  categories:  ship 
performance,  customer  service,  contract  compliance,  and 
human  resources. 

a.  Ship  Performance 

Ship  performance  measures  whether  the  ships  are 
available  when  required,  where  required,  in  a  high  state  of 
material  .readiness  and  at  low  cost .  The  ships  are  required 
to  meet  a  high  operating  tempo  with  no  maintenance  problems. 

Ship  performance  is  measured  by: 

•  Ship  Availability:  Ship  availability  is  directly 
measured  by  the  number  of  survey  days  versus  the 
number  of  survey  days  requested  by  sponsor.  Some 
indirect  measures  to  provide  a  wider  scope  of  ship 


66 


operations  are  percent  of  time  in  meeting  ship 
schedule  goals;  work  progress  of  ships  in  maintenance 
availability,  overhaul  and  equipment  changeouts;  and 
data  provided  from  SORTS,  MOVREPs,  Weekly  Operations 
Summaries  and  Daily  Position  Reports. 

•  Maintenance/Repair:  maintenance/repair  is  indirectly 
measured  by  data  derived  from  SORTS;  CASREPs;  work 
progress  of  ship  in  availability;  equipment  changeout 
and  overhaul;  operating  test  results;  inspection 
results;  product  quality  (sponsor  evaluation  of 
material  condition) ;  and  audit  results  of  ship 
configuration,  SAMM  system,  MRDE  maintenance  and 
propulsion  plant  maintenance. 

•  Exercise  Performance:  This  is  indirectly  measured  by 
how  the  ship  and  crew  respond  to  various  exercises 
and  drills. 

•  Inspection  Results:  These  are  indirectly  measured  by 
an  aggregate  of  administration,  supply  and  security 
inspection  results. 

•  Product  Quality:  A  subjective  measure  derived  from  a 
sponsor  directed  survey  which  asks  for  customer 
evaluations  of  the  four  previous  areas. 

b.  Customer  Service 

Customer  service  pertains  to  those  remaining 
services  that  MSC  provides  directly  to  NAVOCEANO,  or  through 
the  contractor  to  NAVOCEANO,  to  support  daily  ship 
operations.  These  are  no  less  important  than  ship 
performance;  success  or  failure  in  these  areas  will 
determine  whether  one  maintains  or  loses  a  customer. 

Customer  service  is  measured  by: 

•  Fiscal  Responsibility:  This  is  directly  measured  by 
time  to  return  unused  Sponsor  Funds;  percentage  of 
cost  estimates  returned  to  sponsor;  ship  cost  per 
available  day  and  overhead  cost  per  ship.  An 
indirect  measure  can  be  invoice  audit  results. 
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•  Flexibility/Responsiveness:  This  is  directly 
measured  by  the  time  to  respond  to  sponsor  requests. 
Some  indirect  measures  are  customer  satisfaction; 
survey  results  and  validated  user  complaints  from  the 
sponsor . 

•  Logistics:  Logistics  is  directly  measured  by  the 
amount  of  time  to  move  people,  equipment,  mail,  etc., 
to  and  from  the  ship.  This  can  be  indirectly 
measured  by  customer  survey  results. 

•  Port  Services:  This  can  be  indirectly  measured  by 
the  amount  of  time  required  to  meet  customer  requests 
and  by  customer  satisfaction  survey  results. 

c.  Contract  Compliance 

Contract  compliance  monitors  key  areas  to  ensure 
the  contractors  fulfill  all  their  responsibilities  as  set 
forth  in  the  contract.  These  actions  or  inactions,  directly 
or  indirectly,  have  some  impact  on  the  Sponsor.  Measures 
having  to  do  with  maintenance  are  described  under 
Maintenance/Repair . 

Contract  compliance  is  measured  by: 

•  Submission  of  Required  Reports:  An  indirect  measure 
based  on  audit  results  of  whether  all  the  reports  are 
submitted  as  required  and  properly  completed. 

Specific  reports  of  current  interest  to  NAVOCEANO, 
such  as  the  Ship  Sighting  Report,  Mission  Related 
Deck  Equipment  Status  Report,  SAMM  Monthly  Report, 
and  the  Configuration  Change  Report  should  be  tracked 
more  closely  until  the  problems  are  resolved. 

•  Security  Audit  Results:  This  is  an  indirect  measure 
to  ensure  the  contractor  has  met  all  of  the  security 
requirements  he  claims  he  had. 

•  Contract  Audit  Results :  This  is  directly  measured  by 
collecting  contract  audit  results,  including  the 
correction  of  previous  discrepancies. 
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d.  Human  Resources 

Human  Resources  involves  managing  the  contractor's 
mariners  and  MSC's  civilian  mariners  who  operate  the 
oceanographic  ships.  It  encompasses  quality  of  life,  career 
development,  training  and  qualification  requirements,  and 
trustworthiness  evaluations.  All  of  these  items  affect 
NAVOCEANO's  customer  satisfaction.  Finally,  this  category 
includes  quality  of  life  issues  which  not  only  affects  both 
categories  of  mariners  but  also  NAVOCEANO's  survey  crew. 

Human  Resources  is  measured  by: 

•  Quality  of  life:  This  is  indirectly  measured  by 
quality  of  life  surveys  distributed  to  mariners  and 
sponsor  crew  members . 

•  Training:  Training  is  indirectly  measured  by  audit 
results  of  training  and  qualification  records. 

Another  indirect  measure  is  the  civilian  mariner 
(CIVMAR)  training  pipeline  cost. 

•  Career  Development :  Career  development  is  indirectly 
measured  by  CIVMAR  performance /assignments  and  the 
number  of  days  the  mariner  waits  in  the  pool  until 
assignment  to  a  ship. 

•  Trustworthiness  Evaluation:  This  is  directly 
measured  by  the  percent  of  crew  with  completed 
trustworthiness  evaluations  and  number  of  days  to 
process  a  trustworthiness  evaluation  after  receipt  by 
MSC. 

2 .  Internal  Business  Perspectives 

The  internal  business  perspective  measures  MSC's 
internal  operations  that  support  its  customer's 
requirements.  MSC's  internal  measures  should  concentrate  on 
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those  processes  which  have  the  greatest  impact  on  customer 
satisfaction. 

The  internal  business  perspective  is  measured  by: 

•  Processes :  This  is  indirectly  measured  by  such 
things  as  process  cycle  times;  percent  of  MSC  field 
staff  time  spent  on  invoice  certification;  percent  of 
overhaul  jobs  verified  by  MSC  field  staff;  time  to 
review  specification  packages;  percent  invoices 
audited;  MSC  attendance  at  overhauls;  overhead  cost 
per  ship;  and  process  audit  results. 

•  Communication:  This  is  indirectly  measured  by 
internal  survey  results . 

•  Accountability,  Responsibility  and  Authority:  This 
is  indirectly  measured  by  survey  results  from  MSC 
managers . 

•  Workplace  Practices:  This  is  indirectly  measured  by 
internal  survey  results  and  employee  turnover  rate . 

3 .  Innovation 

Innovation  measures  MSC's  ability  to  maintain  and  also 
improve  its  competitiveness.  MSC's  ability  to  innovate  and 
continually  improve  its  processes  will  determine  whether  the 
Special  Mission  Ship  Sponsors  will  continue  to  look  at  other 
ship  management  alternatives,  or  whether  they  will  remain  a 
customer.  In  essence,  without  the  protection  of  Public  Law 
mandating  MSC  as  the  prime  ship  manager,  innovation  is  the 
key  to  MSC's  survival. 

Innovation  is  measured  by: 

•  Cost:  Cost  is  directly  measured  by  percent  reduction 
in  total  cost;  percent  reduction  in  ship  cost  per 
available  day;  and  percent  reduction  in  process  cycle 
time  and  cost. 
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•  Customer  loyalty:  This  is  indirectly  measured  by 
customer  satisfaction  survey  results;  percent  of 
budget  funded  by  new  business;  percent  of  existing 
business  retained;  and  by  the  number  of  RFI's  and 
RFP's  for  ship  services  released  by  sponsors  per 
year . 


4 .  Financial  Perspective 

The  financial  perspective  focuses  on  MSC's  financial 
resources.  These  measures  indicate  how  MSC's  strategy 
affects  its  total  budget  and  the  Sponsor's. 

Financial  perspective  is  measured  by: 

•  Budget :  This  is  directly  measured  by  such  things  as 
time  to  return  unused  sponsor  funds;  percentage  of 
estimated  cost  returned  to  sponsor;  ship  cost  per 
available  day;  overhead  cost  per  ship;  actual 
maintenance  availability  costs  versus  budgets;  and 
monthly  cost  of  ship  operation  versus  budget. 

•  Contract  compliance:  This  is  indirectly  measured  by 
invoice  audit  results  and  percentage  of  total 
invoices  in  error. 


B.  PROPOSED  MEASURES  AND  COMPARISON  TO  CURRENT  MSC 
MEASURES 


The  measurement  structure  for  oceanographic  ship 
management  discussed  above  is  summarized  in  Tables  2  through 
5 .  Those  measures  in  bold  are  the  things  MSCLANT  and  MSCPAC 
are  not  currently  tracking.  It  is  evident,  when  reviewing 
these,  that  there  are  some  proposed  measures  which  overlap 
with  what  MSC  is  currently  tracking.  There  are  also  many 
proposed  measures  which  MSC  is  not  tracking.  A  discussion 
of  this  comparison  appears  in  Chapter  V. 
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Table  2.  Customer  Satisfaction  Measurement  Criteria, 
bold  are  not  currently  tracked  by  MSC. 


Measures  highlighted  in 


CUSTOMER  SATISFACTION 

Ship  Performance 

Ship  Availability 

•  #  actual  survey  days  versus .  #  days  requested 

•  %  of  time  meeting  schedule  goals 

•  Work  progress  of  ship  in  availability,  equipment  changeout 

&  overhaul 

•  MOVREPS 

•  WOSR 

•  Daily  Position  Report 
Maintenance/Repair 

•  SORTS 

•  CASREPs 

•  Work  progress  of  ship  in  availability,  equipment  changeout 

Sc  overhaul 

•  Operating  test  results 

•  Inspection  results 

•  Product  quality  (Sponsor  evaluation  of  material  condition) 

•  Audit  results  (MRDE  maintenance,  ship  configuration,  SAMM 

system,  propulsion  plant  maintenance) 

Exercise  Performance  (ship  &  crew  response  to  drills  and 
exercises) 

Inspection  Results  (an  aggregate  of  admin,  supply  and  security 
inspection  results) 

Product  Quality  (Sponsor  evaluation  of  four  categories  concerning 
ship  performance) 

Customer  Service 

Fiscal  Responsibility 

•  Time  to  return  unused  Sponsor  funds 

•  %  of  estimated  cost  returned  to  Sponsor 

•  Ship  cost  per  available  day 

•  Overhead  cost  per  ship 

•  Invoice  audit  results 
Flexibility/Responsiveness 

•  Time  to  respond  to  Sponsor  request 

•  Customer  satisfaction  survey  results 

•  Validated  user  complaints  from  sponsor 
Logistics 

•  Time  to  move  people/equipment/mail  to  and  from  ship 

•  Customer  satisfaction  survey  results 

Port  Services 

•  Time  to  respond  to  Sponsor  request 

•  Customer  satisfaction  survey  results 

Contract  Compliance 

Submission  of  Required  Reports 

•  Required  report  audit  results 

•  Report  tracking 
Security  Audit  Results 
Contract  Audit  Results 

Human  Resources 

Quality  of  Life  (mariner  and  Sponsor  crew  surveys) 

Training 

•  Training  and  qualification  records  audit  results  (CIVMAR 
and  contractor  mariners) 

•  CIVMAR  training  pipeline  costs 
Career  Development 

•  CIVMAR  performance  /  assignments 

•  #  days  CIVMAR  waits  in  pool  until  ship  assignment 
Trustworthiness  Evaluation 

•  %  of  crew  with  completed  trustworthiness  evaluations 

•  #  days  to  process  trustworthiness  evaluation  after  receipt 

from  contractor 


Table  3.  Internal  Business  Perspective  Measurement  Criteria.  Measures 
highlighted  in  bold  are  not  currently  tracked  by  MSC. _ 

INTERNAL  BUSINESS  PERSPECTIVE 

Processes 

•  Process  cycle  times 

•  %  of  MSC  field  staff  time  spent  on  invoice  certification 

•  %  of  overhaul  jobs  verified  by  MSC  field  staff 

•  Time  to  review  specification  packages 

•  %  invoices  audited 

•  MSC  attendance  at  overhauls 

•  Overhead  cost  per  ship 

•  Process  audit  results 

Communication 

•  MSC  internal  survey  results 
Accountability,  Responsibility  and  Authority 

•  MSC  leadership  survey  results 
Workplace  Practices 

•  MSC  internal  survey  results 

•  Employee  turnover  rate 


Table  4.  Innovation  Measurement  Criteria.  Measures  highlighted  in  bold  are  not 
currently  tracked  by  MSC. _ 


INNOVATION 

Cost 

•  %  reduction  of  total  cost 

•  %  reduction  of  ship  cost  per  available  day 

•  %  reduction  in  process  cycle  time 

•  %  reduction  in  process  cost 


Customer  Loyalty 

•  Customer  satisfaction  survey  results 

•  %  of  budget  funded  by  new  business 

•  %  of  existing  business  retained 

•  #  of  RFIs  and  RFPs  for  ship  services 
released  by  Sponsors  per  year 
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Table  5.  Financial  Perspective  Measurement  Criteria.  Measures  highlighted  in 
bold  are  not  currently  tracked  by  MSC. _ 

FINANCIAL  PERSPECTIVE 

Budget 

•  Time  to  return  unused  Sponsor  funds 

•  %  of  estimated  cost  returned  to  Sponsor 

•  Ship  cost  per  available  day 

•  Overhead  cost  per  ship 

•  Actual  maintenance  availability  / 
overhaul  costs  versus  budgeted 

•  Monthly  cost  of  ship  operation  versus 
budgeted 


Contract  Compliance 

•  Invoice  audit  results 

•  %  of  total  invoices  in  error 


C.  NAVOCEANO  AND  GAO  ISSUES 


This  section  lists  the  issues  that  NAVOCEANO  and  GAO 
have  with  MSC's  ship  management  service  as  previously 
described  in  Section  D  of  Chapter  III.  Following  each 
issue,  measures  are  presented  that  would  identify  the 
problem  and  track  progress  towards  solving  it. 

1 .  NAVOCEANO  Issues 

Problem:  Neglected  maintenance  of  mission  related  deck 

equipment  (MRDE) . 

Measures : 

•  MRDE  maintenance  audit  results 

•  Product  quality  survey  (sponsor  evaluation  of 
material  condition) 

•  Product  quality  survey  (sponsor  evaluation  of  total 
ship  performance) 

•  Submission  of  required  report  (MRDE  Status  Report) 


74 


Problem:  Ship  configuration  in  SAMM  system  is  incomplete 

and  inaccurate . 

Measures : 

•  Ship  configuration  audit  results 

•  SAMM  system  audit  results 

•  Submission  of  required  reports  (SAMM  Monthly  Report) 

•  MSC  attendance  at  overhauls 

Problem:  MSC  overhead  costs  excessive. 

Measures : 

•  Ship  cost  per  available  day 

•  Overhead  cost  per  ship 

•  Number  of  outside  RFQ's  and  RFI's  requested  by 
sponsors  per  year 

•  Customer  service  survey  results 

Problem:  Slow  return  of  unused  sponsor's  funds. 

Measures : 

•  Time  to  return  unused  sponsor  funds 

•  Customer  satisfaction  survey  results 

Problem:  Overstated  cost  estimates. 

Measures : 

•  Percentage  estimated  cost  returned  to  sponsor 

•  Customer  satisfaction  survey  results 

Problem:  Insufficient  Ship  Sighting  Reports  sent. 

Measures : 

•  Submission  of  required  reports  (Ship  Sighting  Report) 
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•  Contract  audit  results 


•  Customer  satisfaction  survey  results 

Problem:  Logistics  problems 

Measures : 

•  Time  to  move  people/equipment /mail  to  ship 

•  Customer  satisfaction  survey  results 

Problem:  Quality  of  Life  (inadequate  food  quality  and 

service) 

Measures : 

•  Mariner/sponsor  quality  of  life  survey  results 

•  Customer  satisfaction  survey  results 

2.  GAO  Audit  Discrepancies 

Problem:  Inadequate  documentation  and  review  of  invoices 

resulted  in  overpayments  and  questionable  pricing. 

Measures : 

•  Percent  of  total  invoices  audited 

Problem:  MSC  field  staff  not  reviewing  invoices. 

Measures : 

•  Percent  of  field  staff  time  spent  on  certification 

Problem:  No  controls  preventing  contractors  from 

circumventing  requirements. 

Measures : 

•  Percent  field  staff  time  spent  on  certification 

Problem:  No  verification  that  overhaul  work  is  complete  and 

reasonably  priced. 
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Measures : 


•  Number  of  overhaul  jobs  verified  by  MSC  field  staff 

Problem:  No  standard  procedures  to  develop  personnel 

requirements . 

Measures : 

•  Percent  of  crew  with  completed  personnel  check 

•  Number  of  days  to  process  requirements 


Problem:  Trustworthiness  evaluations  processed  too  slowly. 

Measures : 

•  Percent  of  crew  with  completed  trustworthiness 
evaluation 

•  Time  to  process  trustworthiness  evaluation  after 
receipt  from  contractor 

Problem:  No  systematic  approach  to  identify  and  implement 

best  practices. 

Comment:  To  the  extent  it  can  be  done,  measurements  in 

general  address  this  issue. 

Problem;  Fragmented  lines  of  authority. 

Comment :  Measurements  are  showing  where,  there  are 

problems  and  will  identify  the  fragmented 
line  of  control  if  that  is  the  problem's 
cause . 


D.  SUMMARY 

The  purpose  of  this  chapter  was  to  propose  a  broad  set 
of  measurement  categories,  along  with  individual  metrics 
which  will  allow  MSC  to  monitor  ongoing  improvements  in  the 
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Special  Mission  Oceanographic  Ship  Program.  These  measures 
were  summarized  in  Tables  2  through  5 .  Those  measures  not 
tracked  by  MSC  at  the  time  of  this  research  were  also 
highlighted  in  the  tables.  Finally,  this  chapter  listed  the 
issues  of  concern  to  NAVOCEANO  and  GAO  with  regard  to  MSC's 
ship  management,  along  with  measures  that  would  identify  the 
problem  and  track  progress  towards  solving  it . 

As  discussed  earlier  in  this  chapter,  there  is  no 
single  correct  way  for  structuring  and  defining  the 
hierarchy  of  measures .  This  is  only  a  proposed  framework 
and  not  necessarily  a  complete  set  of  measures.  Identifying 
and  developing  the  final  measures  for  improvement  must  be 
completed  by  MSC  and  other  cognizant  personnel  who 
participate  in  the  process.  Their  involvement  ensures 
maximum  success  through  acceptance  and  understanding  of  the 
measures . 
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V.  SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY 

This  thesis  set  out  to  develop  a  set  of  measurement 
criteria  to  monitor  the  improvements  made  in  MSC's  Special 
Mission  Oceanographic  Ship  program.  Specifically,  the  goal 
of  this  thesis  was  to  answer  the  following  research 
questions : 

Primary: 

What  appropriate  measures  of  performance  can  MSC  use  to 
monitor  ongoing  improvements  in  the  Special  Mission 
Oceanographic  Ship  program? 

Subsidiary: 

What  aspects  of  service  are  important  to  MSC' s 
oceanographic  ship  customers? 

What  aspects  of  service  does  MSC  perceive  to  be 
important  to  its  customers? 

This  research  revealed  four  broad  measurement  criteria 
that  could  adequately  cover  the  mission  of  the  MSC  Special 
Mission  Oceanographic  Ships.  These  four  include:  customer 
satisfaction,  internal  business  perspective,  innovation  and 
financial  perspective.  They  were  derived  from  the  overview 
of  measurement  criteria,  presented  in  Chapter  II.  Customer 
satisfaction  measures  those  things  which  are  most  important 
to  NAVOCEANO  in  fulfilling  its  mission.  The  internal 


79 


business  perspective  measures  MSC's  internal  operations  that 
support  its  customer's  requirements.  Innovation  measures 
MSC's  ability  to  maintain  and  improve  its  competitiveness, 
and  ensure  its  survival  into  the  future.  The  financial 
perspective  measures  how  MSC's  implementation  of  its 
strategy  affects  its  bottom  line  and  that  of  the  customer. 

The  research  suggests  that  the  aspects  of  service  which 
MSC  considers  important  do  not  match  its  customers'  view. 
Tables  2  through  5  reveal  areas  of  overlap  in  the  measures 
relating  to  ship  availability  and  the  management  of  CIVMARs. 
Unfortunately,  NAVOCEANO's  concerns  extend  beyond  meeting 
operational  and  major  maintenance  schedules.  Research  and 
interviews  revealed  NAVOCEANO  is  also  concerned  about 
managing  scarce  funds,  overhead  costs  and  cost  estimates, 
maintaining  Mission  Related  Deck  Equipment,  logistics, 
security,  and  quality  of  life.  These  gaps  in  MSC's 
measurement  system  appear  in  Tables  2  through  5  as  bold 
print . 

MSC  must  understand  the  mission  and  needs  of  its 
customers,  as  well  as  how  they  define  quality.  A  supplier 
and  customer  may  define  quality  differently.  This  is  why  it 
is  important  to  involve  the  MSC  personnel,  customers  and 
suppliers  in  the  measurement  development  process  to  ensure 
understanding  and  acceptance  of  the  measures. 
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B.  CONCLUSIONS 

This  thesis  research  suggests  the  following 
conclusions : 

There  are  some  elements  of  performance  the  customer 
considers  important  that  MSC  is  not  tracking.  These 
shortfalls  in  service  were  brought  to  light  by  interviewing 
some  MSC  customers,  COMNAVMETOCCOM  and  NAVOCEANO.  They  are 
discussed  in  Chapter  III.  In  these  years  of  tighter  fiscal 
budgets  it  is  increasingly  important  for  MSC  to  create  value 
by  understanding  their  customers'  needs  and  in  filling  those 
needs.  MSC's  reinvention  process  and  implementation  of  a 
program  manager  structure  is  a  step  in  the  right  direction. 

It  is  important  to  get  MSC  personnel,  their  customers 
and  suppliers,  etc.  involved  in  identifying  and  developing 
measures  for  improvement  monitoring.  In  addition  to 
understanding  what  quality  means  to  both  MSC  and  its 
customers,  involvement  by  all  cognizant  personnel  will 
enhance  acceptance  and  understanding,  provide  a  more 
relevant  set  of  performance  measures,  and  maximize 
performance  results.  Once  the  measures  are  developed,  they 
should  be  weighted  (priority  ranking)  based  on  what  is  most 
important  to  both  MSC  and  the  customer.  Using  these  weights 
and  a  uniform  scoring  system,  individual  measures  can  be 
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aggregated  to  reflect  the  Special  Mission  Ship's  overall 
program  performance . 

Measures  identify  problems  and  quantify  the  improvement 
made  in  performance.  A  measurement  system  alone  won't 
correct  problems.  Problem  resolution  generally  requires 
process  innovations  and  a  business  strategy  which  is  linked 
to  the  measures . 


C .  RECOMMENDATIONS 


The  following  recommendations  are  provided: 

•  This  measurement  process  needs  to  be  validated  by  the 
MSC  customer.  This  ensures  understanding  of  how  the 
customer  perceives  quality. 

•  Establish  weights  to  aggregate  measures  (priority 
rankings)  .  MSC  will  gain  an  understanding  of  what  .is 
most  important  to  the  customer. 

•  Expand  this  measurement  process  to  all  MSC  Special 
-Mission  Ship  operations.  The  four  broad  level 

criteria,  and  most  of  the  lower  level  measures  are 
general  enough  for  all  Special  Mission- Program  ships. 
The  specific  measures  can  be  easily  modified  to  fit 
the  specific  equipment  or  situations  applicable  to 
each  sponsor's  mission. 
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APPENDIX.  DEFINITIONS 


These  definitions  related  to  measures  are  provided  in 
support  of  Chapter  IV. 


CASREP 


MOVREP 


MRDE 


SAMM 


SORTS 


Casualty  Report  System.  CASREPs  are  only  submitted 
for  degraded  equipment,  all  others  are  assumed  to 
be  fully  operational.  It  uses  a  rating  scale  with 
C  ratings  of  C-1  through  C-4.  C  ratings  with  a 
higher  suffix  are  worse.  C-4  is  non-operational  and 
affects  ship's  mission. 

Movement  Report .  MOVREPs  are  submitted  upon  the 
ship's  arrival,  departure,  when  a  change  of 
destination  is  made  or  during  Panama  or  Suez  Canal 
transit . 

Mission  Related  Deck  Equipment.  MRDE  consists  of 
winches,  A-frames,  and  other  specialized  deck 
handling  equipment  which  is  used  to  carry  out 
NAVOCEANO's  survey  efforts. 

Shipboard  Automated  Maintenance  Management  system. 
SAMM  is  a  computer  program  used  in  documenting 
shipboard  maintenance  actions.  It  also  contains  a 
database  holding  maintenance  actions  and 
procedures,  shipboard  equipment  configuration  file, 
as  well  as  other  maintenance  related  information. 

Status  of  Resources  and  Training  System.  SORTS 
causes  unit  commanders  to  assess  their  commands  in 
four  areas:  personnel,  training,  quantity  of 
supplies,  and  equipment  condition.  It  utilizes  a 
rating  scale  from  C-1  (fully  capable  in  an  area)  to 
C-4  (not  capable)  . 
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